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1 The politics of social media 
manipulation 


Richard Rogers and Sabine Niederer 


Abstract 

This chapter gives an overview of the contemporary scholarship surround- 
ing ‘fake news’. It discusses how the term has been deployed politically 
as a barb against the free press when publishing inconvenient truths 
since the mid-nineteenth century. It also addresses how such notions 
have been used in reaction to novel publishing practices, including to the 
current social media platforms. More generally, the scholarship could be 
divided into waves, whereby the first related to the definitional issues 
and the production side, whilst the second has been concerned with its 
consumption, including the question of persuasion. There is additionally 
interest in solutions, including the critique of the idea that automation 
effectively addresses the problems. It concludes with research strategies 
for the study of the pervasiveness of problematic information across the 
internet. 


Keywords: fake news, junk news, disinformation, clickbait, hyperpartisan, 
post-truth 


Introduction: Influence campaigning in political spaces online 
and the question of persuasion 


In reviewing the scholarship surrounding so-called fake news, one would 
out of necessity make a distinction between the dominant work on the art 
of influence campaigning and computational propaganda online and the 
consequences to date for its consumers, but also the few findings, often 
journalistic, in the relatively understudied case of the Dutch political 
space online, both on the web as well as in social media. Much work has 
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been undertaken on the influence of Russian (and Russian-style) influ- 
ence campaigning in the US, and the presence or absence thereof during 
elections in Germany, France, Sweden and elsewhere. With respect to 
the Netherlands, the case studies have been reserved to the early Russian 
influence campaigning around the downing of the MH17 Malaysian airliner 
(beginning in 2014) and the suicide bombings in the Brussels airport and 
metro (2016), discovered through the use of Twitter data sets of Russian 
trolls, or influence campaigners. Other work has been performed on the 
existence of home-grown troll networks that are at times ‘pro-Russian’ but 
do not seem to have had foreign input. 

Crucially, in the studies and journalistic treatments to date it is in- 
creasingly remarked that there has been a shift in Russian disinformation 
campaigning from inflaming conflict with the West to stirring it within the 
West. It is also argued that disinformation could be said to be ‘Russifying’, 
i.e., the borrowing of so-called Russian techniques by domestic actors. The 
campaigning, whether foreign or domestic, does more than create narratives 
that divide; it also employs computational means to inflate and amplify 
them through bot work, fake following, astroturfing, the creation of front 
groups and other artificial publicity tactics. 

It is also argued that more attention ought to be paid to the rise of extreme 
and divisive media on social media platforms, where the point is often 
made that great emphasis is being placed on foreign disinformation when 
by comparison it performs poorly in European news spheres. The growth of 
‘hyperpartisan’ news and commentary also may be viewed as an alternative 
fact or knowledge infrastructure, contributing to discussions of a post-truth 
condition and the contention that established institutions are under threat. 

It is of equal importance to examine the critique on persuasion, or 
the extent to which the influence campaigning strategies, artificiality 
and hyperpartisan sources have discernible impacts on their consumers, 
especially the voters. They appear to be minimal. Indeed, there is a small, 
but growing literature critiquing transfer models, also known as hypodermic 
needle or magic bullet theories which themselves could be considered part 
and parcel of the fake news hype and fascinations with so-called psyops 
activities such as in the Cambridge Analytica case.’ Transfer models do 


1 The Cambridge Analytica case or scandal refers to the illegitimate use of over 80 million 
Facebook users’ information to develop micro-targeted advertising (Cadwalladr and Graham- 
Harrison, 2018). It prompted US Congressional and UK Parliamentary investigations, and also 
led to Facebook's tightening its data access for academics and public scrutiny more generally 
(Bruns et al., 2018). 
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nottake into account the active filtering of media users or phatic sharing, 
it is argued, whereby one circulates dubious media more to connect with 
others or for amusement than to pass along substantive information. Such 
models also would discount hardened attitude, and studies finding that 
campaigns generally have minimal effects. 

As for the measures to be taken, the literature both describes and oc- 
casionally questions fact-checking and media literacy efforts because of the 
assumption that corrected information would assuage knowledge deficits, for 
attitudes often remain the same. Nonetheless, among the policy recommen- 
dations most frequently put forward are bolstering media literacy initiatives, 
together with flagging questionable content, manually and automatically, 
for further scrutiny. Social media platforms are facing regulation and are 
asked to address extreme content and create public archives. 

One aspect of the literature review relevant to empirical work concerns 
the methods employed to demarcate political space online for the subsequent 
study of the scope and impact of problematic content, junk news and compu- 
tational propaganda — to use some of the terms for the larger phenomenon 
under study. Under consideration here are largely mixed (quanti-quali) 
techniques and digital methods from media studies and data journalism. 
These provide distinctive political space demarcation strategies for the web 
as well as social media per platform as well as approaches for cross-platform 
analysis. They query engines and platforms, measure significant political 
stories (in terms of engagement) and determine construals of dubiousness 
through news criticism, categorizing significantly engaged-with stories 
into genres such as disinformation, conspiracy, clickbait, hyperpartisan 
and (automated) amplification. While often practiced on a story level, the 
determination of dubiousness also may be made through source criticism, 
according to the extent to which publishers’ output repeatedly accords with 
junk news definitions, discussed in the next section. It is also worth studying 
how signal-based or algorithmic determinations of problematic content 
comport with qualitative methods that are based on source (provenance) 
criticism. 


Fake news, junk news and computational propaganda 


Historically, fake news proclamations could be thought of as linked to 
particular novel publishing practices both ‘when old media were new’ but 
also nowadays through social media platforms (Marvin, 1988; Gitelman, 
2006). The term ‘canard’, meaning unfounded rumour or story, refers to 
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the contents printed in the French broadsheets of the eighteenth century; 
‘scandal sheets’ are the British term for the same era of publishing (Darnton, 
2010). In the U.S., in particular, ‘fake news’ as a term recently experienced 
a revival and travelled internationally, in the numerous senses in which 
it has been deployed historically: ‘news satire, news parody, fabrication, 
manipulation, advertising, and propaganda’ (Tandoc et al., 2018: 137). As 
directed towards contemporary social media platforms, the charge of 
fake news and similar terms often has been uttered as a lament after the 
introduction of new media technologies, where there are concomitant calls 
for new journalistic standards, as witnessed with the competing tabloids 
and their sensationalist, yellow journalism in the late 1890s and into World 
War las well as the radio and newswire in the 1920s (Opper, 1894; Lippmann, 
1922; McQueen, 2018). 

With the rise of corporate public relations, the blurring of the distinction 
between the editorial and the advertisement sent over the wire or into 
the airwaves prompted the use of the moniker, ‘news fakers’ (McKernon, 
1925; Lazer et al., 2018). Similarly, the contents of the early, unedited web, 
populated by self-publishers, and later the blogosphere, were often described 
as ‘too fresh to be true’, given the speed of news production and the potential 
for those looking for a scoop to convert unsubstantiated rumour into news 
(Hall, 2001; Rogers, 2005). More recently, the notion would be routinely 
deployed by satirical news sources such as Saturday Night Live! in the US (Day 
and Thompson, 2012); in fact, The Daily Show, the progressive comedy news 
program, described itself proudly as a ‘fake news program’ (Newman, 2010; 
Harsin, 2018). Parody, it should be recalled, was behind the origination of the 
most circulated ‘fake news’ story during the US presidential campaigns of 
2015-2016, ‘Pope Francis Shocks World, Endorses Donald Trump for President’ 
(Allcott and Gentzkow, 2017). While many definitions concentrate on the 
falseness of content, they may have the ‘trappings of news’ through the use 
of the news apparatus, including the style, formats and containers employed 
(Laquintano and Vee, 2017; Grinberg et al., 2019). Indeed, narrower definitions 
take as their point of departure how the sources ‘falsely claim to be news 
organizations,’ though they may well look the part (Tucker et al., 2018: 3). 

Fake news also has been deployed politically as a barb against the free 
press when publishing inconvenient truths, or speaking ‘truth to power’ 
(Cary, 1955; Darnton, 2017). Since the mid-nineteenth century, labelling 
the news media generally and disagreeable reporting specifically as the 
product of der Liigenpresse or the lying press is a discrediting ploy or even 
communication strategy, still practiced today by far-right social move- 
ments as Pegida in Germany, chanting at street rallies Liigenpresse, halt 
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die Fresse (lying press, shut your mouth) (Beiler and Kiesler, 2018). It was 
the German Unwort des Jahres (notorious word of the year) in 2o14, in the 
competition organized by TU Darmstadt. Fake news is also a label, used in 
highly conservative political circles in the US, for particular news sources, 
notably CNN, MSNBC, The New York Times, and The Washington Post; the 
designation is similar, albeit perhaps more extreme, to past portrayals of 
the agenda-setting ‘elite media’ in contrast to conservative upstarts as Fox 
News (Marwick, 2018; Tripodi, 2018; Peck, 2019). In this respect, one could 
call the current situation just the latest fake news scare, or even moral 
panic (Brennen, 2017; Morozov, 2017). 

When discussing the phenomenon in relation to social media and other 
online sources, researchers at the computational propaganda project at the 
Oxford Internet Institute (OII) often offer the umbrella term ‘junk news’, 
defined as ‘extremist, sensationalist, conspiratorial, masked commentary’ 
(Howard et al., 2017, 1). Other catch-all’s include ‘problematic information’, 
‘information disorders’ and ‘false news’ (Jack, 2017; Wardle and Derakhshan, 
2017). Apart from sensationalist, conspiratorial and masked - features that 
have been a part of fake news ontologies for centuries — the OII defini- 
tion emphasizes another element, extremist, which cuts to the heart of 
contemporary concern for the phenomenon when studied not only as a 
practice of media and public opinion manipulation but also a trigger for 
societal unrest. 

With respect to the growing anxiety over fake news as harbinger of unrest, 
one may refer to the distinctions made between a variety of information 
disorders, as well as the coinage of new terminology that captures excitable, 
Internet-related media and speech (Wardle, 2018). First, disinformation and 
misinformation are both false, but the latter is unintentionally so, whilst the 
former is fashioned for the purposes of intentional disruption and causing 
harm. A third term, ‘mal-information’ (a neologism), seemingly borrowed 
from malware or malicious software categorizations, has been introduced 
to describe accurate information released for the purposes of harassment 
such as doxing, or publishing private details (Wardle and Derakhshan, 2017). 

These are the tools for the so-called ‘weaponization’ of social media 
platforms to foment discord through seeding the news and public opinion 
with divisive content. Indeed, ‘extreme speech’ is a term that has been offered 
as a nuancing of the hate speech discourse as it is applied to online toxicity. 
It is meant to capture a form of charged language and cultural conflict that 
stops short of hate, and has emerged with social media, defined as ‘vitriolic 
exchange on Internet-enabled media’ (Pohjonen and Udupa, 2017: 1173). 
Its rise has prompted social media companies as Facebook, Twitter and 
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Alphabet (owners of YouTube) to expand their content reviewer pools as 
well as widen their internal mandates to identify and remove more than 
violence, pornography and hate (Gillespie, 2018). Google also installed a 
feedback system for its web search to report inappropriate autosuggestions, 
after reports of queries for the ‘holocaust’ autocompleting with ‘is a hoax’ 
(Solon and Levin, 2016; Hern, 2017). 

As with new media technologies of old, social media platforms currently 
are said to enable the ‘supercharging’ or the acceleration of the spread of 
fake news (Bounegru et al., 2018). Two terms have been used to capture the 
web and subsequently social media as accelerationist media: clickbait and 
computational propaganda. Clickbait connotes titillating and sensational 
content and is formulaic in its presentation, often containing numbered lists 
(sometimes referred to as a ‘listicle’) as well as a cliff-hanger or ‘information 
gap’ that sparks curiosity, e.g., ‘twenty things you should not do when visiting 
Japan’. Facebook, in seeking to identify and downgrade clickbait in its news 
feed, defines it as ‘a posted link with a headline that encourages people 
to click to see more, without telling them much information about what 
they will see’ (O’Donovan, 2018). Generally social media companies seek to 
operationalize substantive definitions into computational signals. Thus, to 
Facebook, brief attention (or short ‘time-on-site’) is a signal of clickbait, for 
readers, having been lured in to the ‘junk food of content consumption’, are 
subsequently dissatisfied with the low-quality content, and leave the page 
quickly (DeAmicis, 2014). Clickbait, often innocuous, can be combined with 
divisive content (Burger and Schenk, 2019). ‘Extreme clickbait’ was a part of 
the story behind the allegedly apolitical Macedonian teens based in Veles, 
who used ‘spammy techniques’ in optimizing pro-Trump sites to make 
money, in the run-up to the US presidential elections of 2016 (Silverman and 
Alexander, 2016). Follow-up reporting has sought to debunk that narrative, 
finding that the clickbait campaign was orchestrated by political operatives 
(Wendling, 2018; Silverman et al., 2018). 

Computational propaganda, the second term, refers to ‘the assemblage of 
social media, autonomous agents and algorithms tasked with the manipula- 
tion of opinion’ (Neudert, 2017: 3). The breadth of the definition is intended 
to capture the bots that amplify content, the advertising platforms that 
enable micro-targeting and personalization of influence messaging, and 
the click farms that inflate the follower counts and engagement scores, 
granting posts higher ‘vanity metrics’ and thus greater symbolic power 
through fake support (Rogers, 2018a). For computational propaganda, bots 
increase the spread or reach of the posts and inflate their metric counts 
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(Woolley and Howard, 2016). ‘Low-credibility content’ is spread dispropor- 
tionately by ‘social bots, which refer to bots or autonomous agents tasked 
with influencing discussion and public opinion; such a finding has led to 
calls for curtailing their use (Shoa et al., 2018). As a part of the ‘assemblage’ 
of actors and software practicing computational propaganda, the work of 
software-assisted, political operatives has come under scrutiny, especially 
in the run-up to elections. Sock puppets, assuming the false identity of a 
grassroots organizer or a concerned individual, create and circulate political 
content, organize events and mobilize audiences, making interventions in 
the physical world through hashtags, internet memes and Facebook events 
(Mina, 2019). ‘Front groups’ or even faux ‘hashtag publics’ also mobilize 
followings and organize demonstrations (see Table 1.1); one notorious case 
concerned an anti-Islam protest and counter-protest in Houston, Texas, in 
2016, where both groups were mobilized by Russian campaigners operating 
under the names of the Blacktivists and the Heart of Texas, respectively 
(Shane, 2018). 

A related term for disingenuous content insertion for political ends is 
astroturfing. It is the artificial seeding of newspapers and other content 
providers with political (or corporate) advertising disguised as genuine 
citizen concern. Such content is a different category than sponsored political 
content, where there are regulations that mandate labelling it as ‘paid for 
by’ a particular candidate or campaign (Vaidhyanathan, 2017). Nonetheless 
there have been calls to have ‘masked’ political content unmasked and 
marked as sponsored, however much in the case of a pro-Brexit group, 
Britain’s Future, investigative journalists were long not able to unearth the 
funding source, despite the transparency of its being labelled. 

Particular forms of native social media advertising have prompted 
the calls for further public scrutiny of political ads, and also perhaps an 
expansion of the definition of such. ‘Dark posts’ (aka ‘promoted posts’) 
on Facebook refer to micro-targeted advertisements, without a referral 
page anchoring the content for further investigation (Bump, 2017). Used by 
political operatives, including foreign influence campaigners, in the US in 
2014-2017 and beyond, such campaigning tactics assemble ‘keyword publics’ 
algorithmically by querying the Facebook advertising platform for words 
such as ‘second amendment’ or other pro-gun terminology and sending 
advertisements to the news feeds of the tens or hundreds of thousands of 
those users determined to have such an interest (Angwin et al., 2017). These 
publics are targeted not so much because they are persuadable voters but 
rather to have them circulate and amplify messaging. 
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Table 1.1 Overview of 2016 fake rallies planned and promoted, as listed in the 
US indictment of 13 Russian nationals concerning foreign election 
interference 


2016 fake rallies planned and promoted 


Date Fake rally Location 

25June March for Trump New York 

9 July Support Hillary. Save American Muslims Washington, D.C. 

23 July Down with Hillary New York 

20 Aug. Florida goes Trump Several Florida cities 

2 Oct. Miners for Trump Several Pennsylvania cities 
12 Nov. Show your support for President-Elect Donald Trump New York 

12 Nov. Trump is NOT my president New York 

19 Nov. Charlotte against Trump Charlotte, N.C. 


Source: Parlapiano and Lee (2018) 


Apart from particular social media advertising products such as dark posts, 
other formats have been identified as energizing publics with divisive 
messages. “Image macros’, also known as memes, are photos with two lines of 
text, one opening and one closing line, that are a popular format for political 
messaging on Facebook and have been among the most shared and otherwise 
most engaged-with content on the platform (Renner, 2017). Indeed, in the 
data analysis of the most shared posts of the ‘fake’ (or astroturfing) activist 
group pages set up by the Russian Internet Research Agency (Blacktivists, 
United Muslims of America, Being Patriotic, Heart of Texas, Secured Borders 
and LGBT United), the image macros and other meme material scored 
particularly well (Chen, 2015; Albright, 2017; Timberg, 2017). 


Russian influence campaigning, Russification and the 
‘hyperpartisan’ style 


‘Dark globalization’ is a term put forward by the historian Timothy Snyder to 
refer to how knowledge of western societal problems provides opportunities 
to influence campaigners from abroad, or Russia in particular (2018). In the 
US Snyder refers to the complex of race, gerrymandering and the electoral 
college, and the capacity to target voters in specific geographical areas (such 
as counties in ‘swing states’) with divisive political messaging that amplify 
or provide ‘oxygen’ to viewpoints. There have been detailed analyses of the 
Russian influence campaign of 2014-2017 commissioned by the US Congress, 
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both of which benefited from data provided by Facebook, Twitter and 
Alphabet (Google) that previously had not been made available for research 
(Howard et al., 2oi8; New Knowledge, 2o18). They are a part of a litany of 
literature that has appeared since the commissioning by governments to 
study the ‘tactics’ of the influence campaigners as well as the contemporary 
art of propaganda and the development of counter-narratives more generally. 
These studies also have led to recommendations concerning how to combat 
the effects. 

The study by the cybersecurity firm, New Knowledge, emphasizes the 
collective cognitive dissonance that effective propaganda achieves, introduc- 
ing (and popularizing) language from intelligence and counterintelligence 
work (2018). Among the goals of the propagandists is to create ‘a wilderness 
of mirrors’, originally a phrase from a T.S. Eliot poem but mobilized by the 
intelligence community (Holzman, 2008). It refers to an environment where 
truth (and its establishment) are no longer self-evident (Groll, 2018). 

To achieve that goal, New Knowledge argues, one particular tactic is the 
creation of a similarly termed ‘media mirage, or ‘interlinked information 
ecosystems designed to immerse and surround targeted audiences’ (2018: 42). 
They are enveloped in an ‘information cacophony’, where stories from the 
press are repurposed, and given another author (‘blacktivists’), interpretation 
and tone. Here is one example, taken from an original newspaper story about 
how an ‘11-Year-Old Texas Boy Invents Device to Prevent Hot Car Deaths’ 
(Dahlgren and Arkin, 2017). It was reworked as follows: ‘[T]hese are stories 
of Black children the media don’t want you to see’; ‘White people invent 
tools for killing, this Black child is inventing a tool for saving lives’ (New 
Knowledge, 2018: 62). The divisiveness and the effectiveness ascribed to 
the sample post derives not only from the transformation of the feel-good 
news story into a contrived in-group and out-group divide based on race. 
Note, too, the format used; the second sentence is a two-liner, to be cast 
into an image macro or meme, the popular format for sharing and further 
circulation of grievance, outrage as well as mockery. The story also brings 
together categories of problematic information. It is both clickbait as well 
as rather extreme content, and it invites the consumer to read more about 
the grievance. It is also packaged to be shared. 

The purpose of such campaigning is to sow discord and enmity, but it is 
only one of a variety of tactics where the overall goal is to remove a sense of 
a collective and shared experience of the world, as analysts have phrased it, 
and reify group formation (Gessen, 2018). Apart from the creation of a media 
mirage, the other tactics listed are as follows: ‘targeting, asset development, 
cross-platform brand building, memetics, inflecting a common message for 
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different audiences, narrative repetition and dispersal, repurposing and 
re-titling pages and brands, manipulating journalism, amplify conspiratorial 
narratives, sow literal division, and dismiss and redirect’ (New Knowledge, 
2018: 2). With respect to social media, as discussed above, targeting could 
refer to the audience segmentation available in platforms for advertising 
purposes, and memetics to the use of both the image macro to formulate 
a punchy message as well as to build the meme as an additive content 
container for narrative reinforcement. 

It is worthwhile to mention that the expert studies are snapshots, but 
these as well as subsequent reporting have pointed to the ‘ongoing efforts’ 
of the influence campaigners, and their global spread. While social media 
companies — since the Cambridge Analytica and fake news scandals — have 
become more active in identifying and suspending accounts of known Rus- 
sian and other state-sponsored trolls (e.g., Iranian), similarly named accounts 
are active and can be traced to known networks of political operatives 
(New Knowledge, 2018; FireEye, 2018). New accounts are continually made 
(Vaidhyanathan, 2018); the Chief Technology Officer at Facebook speaks 
of ‘blocking more than one million fake accounts every day, sometimes 
just when they are created’ (O’Brien, 2019). The percentage of influence 
campaigner accounts in that large number is not known. 

Recently, there has been growing concern not only about the ongoing 
efforts of Russian influence campaigners but also the uptake by other groups 
(or ‘domestic actors’) of the so-called ‘Russian playbook’ (Frenkel et al., 
2019). Journalistic coverage was prompted by the announcement by Twitter 
that prior to the US Congressional elections of 2018 it removed accounts 
of Americans posing as members of state Republican parties (Harvey and 
Roth, 2018). Facebook also announced that hyperpartisan pages on both 
sides of the political spectrum in the US would be removed. Discussions 
of the ‘Russification’ of online political campaigning also historicized 
disinformation, pointing to the classic examples, such as the claim that 
the HIV virus was the leaked product of a US bioweapons lab; it was planted 
in news outlets beginning in 1983 by Soviet dezinformatsiya campaigners 
in ‘Operation Infektion’ and ultimately spread four years later to national 
US TV news (Boghardt, 2009; Ellick and Westbrook, 2018). Comparing the 
time span of such news spread to the dynamics of reach in the hybrid media 
system nowadays is how one may describe how the ‘platform press’ has 
supercharged fake news (Chadwick, 2013; Bell and Owen, 2017). 

In a well-cited article in the New York Times, Facebook, as a leading 
example of the ‘platform press’, was described as a ‘totally insane, unin- 
tentionally gigantic, hyperpartisan political-media machine’ (Herrman, 
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2016). The author spends some time describing the manner in which 
Facebook mixes posts in its news feed from both family members and 
faint acquaintances, but also discusses the presence of upstart media 
organizations and self-styled advocacy groups that only exist online, many 
only in social media. Most are described as ‘hyperpartisan’. These sources 
populating the platform with content are defined as ‘openly ideological 
web operations’ (Herrman, 2016). They also are successful, not just because 
more extreme and sensational content spreads faster than more sober- 
ing truth (Vosoughi et al., 2018). It is also because they employ formats 
that engage large numbers of users and learn from their engagement and 
reach. ‘Operating’ in a continuous feedback loop of metrics data, posts are 
optimized to perform well in social media. The performance measures are 
based on the virality of posts, and those that work well are emulated. There 
are particular formats as well as styles that drive engagement. Memes and 
clickbait such as listicles, cliff-hanger headlines and human-interest stories 
are among the formats used, as mentioned above. The hyperpartisan style 
has a variety of substantive features, not all of which are equally applied, 
but many appear to be working well. Often anti-establishment as well 
as positioned as against or in competition with the truth-seeking and 
fact-finding of the mainstream media, the media operations post stories 
that are alternatives. These alternatives may be interpretations, facts and 
editorial commentary on events. They become media layers on the news. 
The presentation is often edgy, both in terms of being knowledgeably on 
trend but also sharp in tone. The posts are regular, and as such are part of 
the permanent updating culture, providing a competing ‘feed’ about what 
is happening in the world and in media. 


The post-truth condition 


There is a series of contemporary utterances that have contributed to public 
discourse about a post-truth condition. One is the satirical notion of ‘truthi- 
ness’ (Colbert Report, 2005). Developed as political news commentary 
and comedy, it refers to having the appearance of being true, but without 
evidentiary basis. Another — ‘alternative facts’ — is a term that initially 
referred to the insistence by a member of the US Trump administration that 
the number of attendees at the presidential inauguration in 2016 was higher 
than reported and measured by crowd science (Still, 2017). The subsequent 
clarification of the meaning behind ‘alternative facts’ is more to the point: 
‘additional facts, alternative interpretation’ (Nuzzi, 2017). Compared to 
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truthiness, here facticity does not derive from eye-witnessing or additional 
methodological authority but rather from other fact-making. 

In response to what is sometimes considered first-order objectivity battles, 
or disputes over matters of fact (Margolis, 1995; Latour, 2008), newspaper 
reporting with such headlines as ‘Here Are the Real [Facts]’ as well as the 
work by fact-checking bureaus and initiatives are contesting fact claims with 
increasing urgency (Fandos, 2017). These are public debates about facts, in- 
puts into which include fact-checking, a common practice ofjournalists and 
university research groups seeking to confirm the basis behind particular 
statements by politicians and others (Graves, 2016). Recently, scholarship on 
the effectiveness of fact-checking has developed in at least two directions: 
the extent to which fact-checking corrects the record as well as factual 
beliefs, and whether it changes attitudes (Barrera et al., 2017). Both are part 
of the decades-long discussion and critique of the ‘information deficit’ and 
‘diffusion’ models, which challenge ideas that providing correctives clears 
up controversies (Wynne, 1991; McNeil, 2013). 

In the fake news scholarly discourse, it has been found that there are 
distinct audiences for ‘alternative facts’ and ‘fact-checked facts’ (Bounegru 
et al., 2018). Whilst there may be a correction to the record, the original 
audience may not have been exposed to it. Fact-checked stories also have 
similar circulation patterns to alternative facts; they are forwarded to 
like-minded audiences (Shin and Thorson, 2017). Though it does not tell the 
entire story about exposure, both the original as well as the fact-checking 
publications are outlets with distinctive audiences or subscriber bases, with 
fact-checking newsletters often with smaller, specialty circulations, though 
their visibility may increase as they are built into platform interfaces such 
as Facebook’s. In the other strand of work, it is asked, does exposure to 
fact-checked facts change factual beliefs as well as attitudes? Here one set of 
findings is in keeping with the critiques of the effectiveness of fact-checking 
and the information deficit model more generally, for respondents saw their 
factual accuracy improve, but their attitudes remain unchanged (Nyhan et 
al., 2019). Fact-checking, however, could be understood as a documenting 
process that corrects the record by capturing a dubious story and committing 
it, and its debunking or exposure, to searchable databases and other media. 

The post-truth condition, though, has been described as a competition with 
respect to not first-order but second-order objectivity. In such a circumstance 
there is a rise of competing regimes of truth (Fuller, 2018). Expertise becomes 
‘sectarian’ (Turner, 2001). The idea of the media mirage (evoked to describe 
effective disinformation campaigns) does not in itself create a competing truth 
regime or infrastructure. Rather, it introduces noise into an infrastructure. 


THE POLITICS OF SOCIAL MEDIA MANIPULATION 31 


But when propagandists, or in a different reading of the contemporary situ- 
ation, a populist radical right media ecology, create an alternative news and 
information infrastructure, those efforts fit with descriptions of the post-truth 
condition (Benkler et al., 2017; Sängerlaub et al., 2017). 

In other words, post-truth is a term that should not be construed as 
signifying hoodwinked (or radicalized) consumers, or the ‘wholesale 
cheapening’ of fact-making (Sismondo, 2017). Rather, in asking whether 
‘we can have our facts back’, the debate concerns whether (or when) publics 
can agree on the ‘facticity infrastructure’ or even the modernist project of 
knowledge institutions (Marres, 2018). As a case in point, there are ideologi- 
cally distinctive alternatives to Wikipedia (such as Infogalactic, Metapedia 
and Conservapedia), producing encyclopaedias challenging not only what 
is known or settled fact, but also the sources rooting it (Fitts, 2017). 


Elections, disinformation, and the Dutch case 


Three recurring topics are often discussed in the news and (commissioned) 
research on disinformation and fake news in the Dutch context. First of all, 
of particular concern are Russian trolls and their spreading of disinformation 
in the Netherlands. Secondly, there are the (non-Russian) fake accounts 
and fake fans that that inflate the popularity of a campaign or a prominent 
figure, granting them greater symbolic power. And thirdly, publications 
are addressing its discernibility and possible countermeasures. How to 
recognize it and combat it? Each of these discussions is often set against 
the backdrop of a changing news media landscape, whereby mainstream 
news is increasingly competing with more tendentious and hyperpartisan 
outlets, and digitization is leading to user-driven and algorithm-driven 
personalization. That may narrow the horizon of news that users encounter 
and perhaps increase fringe consumption, though in empirical studies 
such has not been found (Wieringa et al., 2017). Comparisons of the Dutch 
situation are also drawn with the US. 

While digitization may be changing how people consume news, a study 
of online news behaviour, disinformation, and personalization of the news 
by the Rathenau Institute stresses that in the Netherlands, the traditional 
news media still hold a firm and stable position in the media landscape (van 
Keulen et al., 2018). The study also finds that there is not (yet) widespread 
algorithmic personalization in Dutch media sites. And, in stark contrast to 
the current situation in the US, Dutch news consumers tend to use a variety 
of sources and have trust in the traditional news media (and less so in social 
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media). Lastly, the report underlines that the Netherlands does not have 
sucha particularly polarized media landscape as the US. 


Overall, there is a strikingly moderate tone of voice in the literature on 
the Dutch case, both in news reporting and research reports. Since 2016, 
severalstudies have looked at disinformation practices in the Dutch political 
landscape, and each of them has concluded that neither is there anylarge- 
scale disinformation activity in the Dutch media nor does disinformation 
have a significant impact on Dutch citizens. However, in the Summer of 2017, 
Wilfred Rietdijk, a Dutch general and national security advisor, announced 
in an interview with Dutch newspaper de Volkskrant that the Netherlands 
could no longer deal with the digital threat (van Zijl and Modderkolk, 2017). 
A ‘landslide of fake news’, as the subsequent tabloid headline read, would 
lead the country into chaos and division (Jonker, 2017). Including breaches 
and intrusions in his threat assessment (thereby widening the scope beyond 
disinformation), Rietdijk explained how Dutch companies are ‘in the line 
of fire’ from ‘thousands of hackers from Russia, China, and countries such 
as Iran and even Sudan’ (van Zijl and Modderkolk, 2017). The general is not 
the first to warn of foreign interference in the Dutch online space, though 
case studies were lacking, at least in the public domain. 


Russian trolling and its perceived insignificance in the Netherlands 


When the Minister of Internal Affairs, Kajsa Ollongren, warned the Dutch 
government of Russian disinformation in the Netherlands, she initially was 
criticized for not having compelling examples (Pleijter, 2017; Kist and Was- 
sens, 2018a). Two journalistic studies that have looked into Russian tweets 
have found activity in the Dutch online realm, however. A study by NRC 
Handelsblad mined 200,000 tweets from Russian Internet Research Agency 
(IRA) accounts and found disinformation campaigning beginning in 2014 
and another spate in 2016. The weekly magazine De Groene Amsterdammer 
combined the NRC Handelsblad data with larger collections of Russian troll 
accounts, made available on the American public opinion analysis website, 
FiveThirtyEight as well as the lists published by American Congress (van 
der Noordaa and van de Ven, 2018a). Both studies found a peak in trolling 
activity after the downing of MH17 in July of 2014. The NRC Handelsblad 
study finds that Russian trolls posted 57,500 tweets, most of which were 
in Russian and aimed to influence public opinion in Russia and Ukraine, 
and only four of the tweets were in Dutch (Kist and Wassens, 2018b). The 
study by De Groene Amsterdammer confirms that most tweets on MH17 
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were in Russian but finds more mentions of Dutch “conspiracy theorists 
and activists’, indicating a shift from challenging Western narratives (for 
Russian-speaking audiences) to seeking to stir conflict within the West. 

A second event revealed more coordinated Russian troll activity in the 
Dutch language Twitter space (in Belgium and the Netherlands), and a 
further example of striving to foment unrest, albeit unsuccessfully (accord- 
ing to engagement measures) (van der Noordaa and van de Ven, 2018b). It 
concerned the spreading of anti-Islam content directly following the terrorist 
attacks in the Brussels airport and metro in March 2016, and in the two years 
after the attacks. This anti-Islam ‘campaign’ involved about 950 tweets in 
the Dutch language that were circulated by some 150 IRA-related accounts. 
These tweets were rarely retweeted, however. In the event, Russian trolls are 
more successful in the Netherlands with the circulation of English-language 
content. While these tweets are not related to Dutch issues and focus on for 
instance the US elections, they have been shared widely by over 6,000 Dutch 
Twitter users with a total of 9.5 million followers (Kist and Wassens, 2018a). 

Perhaps counterintuitively, there was only minimal Russian interference 
with the Ukraine referendum in the Netherlands in April of 2016 (NOS, 2017). 
There was the Russian video capturing fake Ukrainian far-right militia 
members threatening terrorist attacks in the Netherlands and burning a 
Dutch flag, but it was readily recognized as propaganda (Bellingcat, 2016). 
Otherwise, only a handful of tweets propagating a ‘No’ vote was found in 
the larger set of tweets under study (van der Noordaa and van de Ven, 2018a). 

The NRC Handelsblad concludes its work on the Twitter data set by noting 
that it is possible there is larger scale Russian activity in the Netherlands; it 
should be studied beyond just Twitter to include other platforms with known 
troll activity, such as Facebook, Instagram, YouTube and Reddit. Indeed, 
especially after Trump’s victory in the US presidential elections of 2016, many 
news outlets pointed towards Facebook. As discussed in some detail below, 
a study by BuzzFeed News compiled the most engaged-with posts in the nine 
months prior to the elections and found that so-called fake news during 
that time was circulating more than mainstream news. Journalists from the 
NRC Handelsblad replicated the study’s general method for the Netherlands, 
but with a narrower definition of fake news. They determined that the one 
hundred most-shared political news articles from January and February of 
2017, in the run-up to the Dutch general elections, did not contain fake news 
(Kist and Zantingh, 2017). Certain articles could be considered misleading 
or biased, they thought, for they exaggerated news facts or took them out of 
context. The themes that were most resonant during the campaign period 
in the Netherlands were immigration, Islam and Geert Wilders. 
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Dutch fake followers and trolls 


Until November of 2017 much of the reporting has insisted that the Nether- 
lands — and the Dutch elections in particular — have been largely unaffected 
by disinformation or fake news. Much of the news coverage that speaks of 
it concerns ‘fake followers’. For instance, in 2015, there was a small scandal 
about Geert Wilders concerning a dubious increase in his followers on 
Twitter. Indeed, when Twitter addressed the issue of fake followers and 
follower count inflation through a mass removal of suspect accounts in 
2018, Wilders as well as other Dutch politicians (including from the political 
party Denk) saw their metrics decline (NOS, 2018). In perhaps the most 
well-known case, the Dutch singer-songwriter Dotan was found to have a 
fake following of 140 user accounts, which were used between 2011 and 2017 
to like the musician on social media, edit the Wikipedia article on the artist, 
request his songs at radio stations and circulate heart-warming stories about 
him across social media platforms. One of the profiles declared how Dotan’s 
music helped her through a period of grief after a miscarriage; another tells 
how Dotan welcomed one fan’s terminally ill brother in a meet-and-greet, 
throughout which the singer held the boy’s hand. Both testimonials were 
false, as reporters of de Volkskrant found and Dotan later confirmed (Misérus 
and van der Noordaa, 2018a; 2018b). 

In 2018 the first large-scale global study of computational propaganda 
was published, examining organized social media manipulation such as 
the use of fake followers in 48 countries, including the Netherlands (Brad- 
shaw and Howard, 2018). The study describes the different computational 
tactics employed not so much by Russian influence campaigners but by 
political parties to influence voters and the elections.” It was found that 
the use of social media as an infrastructure for the spread of propaganda 
and disinformation has become widespread. Under examination is ‘cyber 
troop activity, defined as ‘government or political party use of social media 
to manipulate public opinion’ (Bradshaw and Howard, 2018: 9). 

While in more authoritarian regimes, social media manipulation fits 
into larger scheme of voter suppression and election rigging, in ‘emerging 


2 The research conducted a content analysis of news articles reporting on cyber troop activity 
in a sample of 48 countries, supplemented by an in-depth secondary literature review. To collect 
the news articles, the researchers used the following keywords in combination, in queries across 
Google, Yahoo!, Bing and LexisNexis: astroturf*; bot; Cambridge Analytica; Facebook; fake; 
fake account; disinformation; government; information warfare; intelligent agent; military; 
misinformation; persona management; pro-government; propaganda; psychological operations; 
psyops; social media; sock puppet”; troll*; Twitter (2018: 8). 
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and Western democracies, sophisticated data analytics, and political bots 
are being used to poison the information environment, promote scepticism 
and distrust, polarize voting constituencies,and undermine the integrity of 
democratic processes' (Bradshaw and Howard, 2o18: 5). The tactics described 
include the use of three kinds of fake accounts. First, there is the creation 
ofonline commentator accounts that attack and troll genuine users, spread 
divisive content, or [divert] conversations or criticism away from important 
issues’ (Bradshaw and Howard, 2018: 11). A second tactic entails automated 
accounts or political bots to automatedly flood particular hashtags, and 
astroturf by faking a follower base. The bots also troll genuine users by 
reporting them and flag organic content thereby having both suspended 
until a human moderator checks them. A third tactic is the use of hybrid 
accounts, which are those that make use of automation (for the sake of speed 
and convenience) but are actively curated by human users, who commonly 
manage multiple fake accounts or sock puppets. This type of fake account is 
difficult to recognize, and thus to combat. The study finds that automation 
is the most recurring tactic, seen in 38 of the 48 countries under study. 

Besides fake accounts, other strategies involve the use of political ads 
and the involvement of search engine optimization and activity on chat 
applications and across social media platforms. Where Twitter is proven to 
be the platform most friendly to automation, the study finds ‘cyber troop 
activity on chat applications or other platforms (Instagram, LINE, SnapChat, 
Telegram, Tinder, WeChat, WhatsApp} in one-quarter of the countries under 
study (Bradshaw and Howard, 2018: 13). In the European countries in their 
sample, they find distinct junk news footprints per country. In Germany, 
it is rather marginal and was mostly circulated by far-right political actors 
during the 2017 federal elections. In Italy on the other hand, a large and 
active ‘ecosystem’ of it is connected to political forces such as the Lega 
Nord (Northern League) and the Movimento Cinque Stelle (Map, 5 Stars 
Movement), which were at work during the 2017 constitutional referendum 
and the elections of 2018. Here, junk news connects national politics to 
Euroscepticism, conspiracy theory, aliens and pro-Putin propaganda. In 
the Netherlands, the analysis finds that it revolves around politician Geert 
Wilders and in particular the spread of his anti-Islam video, which was 
broadcast on television and shared in social media in the lead-up to the 
2017 Dutch national elections. In particular, the study finds that automated 
accounts have amplified Geert Wilders’ campaign hashtags. 

These results match the findings in a study that looked at troll-like 
behaviour on Twitter, leading up to the 2017 Dutch general elections, where 
sock puppets were found (Bounegru et al., 2018). The study collected over 
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500,000 tweets mentioning at least one of the Twitter accounts of the 28 
political leaders a month before the 2017 Dutch general elections. To retain 
the users that demonstrated troll-like behaviour, it narrowed down the 
set to only the 25 users who consistently targeted one or more political 
representative. The analysis showed that there was a notable asymmetry in 
the distribution of targets of troll-like behaviour and sock puppetry across 
the political spectrum, where left-wing politicians are most often targeted 
by negative mentions, while right-wing politicians receive support. Troll 
content extended to reputable news sources which cited it at least thirty 
times. Among the cited troll accounts were fake news organizations with 
names as ‘Today in Syria’ and “WorldNewsPolitics’, political parties (including 
multiple fake accounts for the Republican party in Tennessee) and concerned 
citizens, most of whom were fiercely pro-Trump and anti-Islam (Kist and 
Wassens, 2017). In another analysis by the NRC Handelsblad, a Dutch political 
party (DENK) also exhibited troll-like behaviour, including sock puppetry 
on both Twitter as well as Facebook (Kouwenhoven and Logtenberg, 2017). 

While Dutch news consumers have been found to use a variety of news 
sources, the Netherlands also has a steady ‘pulp news’ diet (Burger et al., 
2019; van der Poel, 2019). From 2013-2017 Dutch Facebook users consumed 
more low-quality, commercially driven clickbait than mainstream news, 
as was found through engagement scores. As may be expected, there is also 
relatively more clickbait on Facebook than quality news. 

The consumption and forwarding of clickbait, extreme clickbait as well 
as other problematic information extends also to politicians and public 
figures. One Dutch researcher, Peter Burger, has a collection of instances 
when Dutch politicians have retweeted anti-Semitic or otherwise disturbing 
content. In one example, a video purporting to show ‘Muslims vandalizing 
Christmas market in Lithuania’ was actually a recording of an event that 
took place in the city of Baltimore in the US (Burger, 2016). 


Recognizing and countering disinformation in the Dutch online space 


Various initiatives aim to detect and counter disinformation in the Neth- 
erlands and on an EU-level. The EU taskforce (East Stratcom Task Force) 
against disinformation was heavily criticized in the Netherlands after its 
project EUvsDisInfo mistakenly categorized articles by The Post Online, 
GeenStijl and De Gelderlander as disinformation (van Keulen et al., 2018; 
Heck, 2018). (Figure 1.1 shows a cartoon about the fake news taskforce, 


3 By @mentioning them at least 100 times in a one-month period. 
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Figure1.1 Cartoon that ridicules the fake news taskforce, stating: ‘internet trolls 


are best countered by internet hobbits’ 


FOKKE & SUKKE 


HEBBEN ZITTING IN PE TASKFORCE 
NEPNIEUWS 


INTERNET-TROLLEN 
BESTRIJP JE HET BEST... 


NET INTERNET- 
HOBBITS! 


RGvT 


Source: Reid et al. (2018) 


stating internet trolls are best countered with internet hobbits.) In a sense, 
the dispute stemmed from misreadings of single stories, perhaps without 
an appreciation of how settled some of the sources are in the Dutch media 
landscape, despite their tendentious style (in the case of The Post Online 
and GeenStijl). For its part, De Gelderlander had taken over nearly verbatim 
a Russian storyline concerning the perpetrator behind the downing of the 
MH17 but did attribute it to its original source in a barebones reporting style. 
The flagged cases were removed from the EUvsDisInfo site after complaints 
by the Dutch media organization Persgroep (EUvsDisinfo, 2018). 
Fact-checking as a journalistic practice has taken hold in the Nether- 
lands. Many newspapers have implemented (or revived) a fact-checking 
section, often dedicated to checking statements made by political figures 
in interviews in newspapers or TV shows. There are also websites such as 
Hoaxmelding.nl and Nieuwscheckers.nl that compile lists of instances of 
false news on Facebook and elsewhere. For their study of disinformation, 
Rathenau researchers analyzed these lists, comprising respectively 140 on 
Hoaxmelding (collected between 1 February 2014 and 18 December 2017) and 
166 on Nieuwscheckers (between 3 February 2017 and 5 January 2018) (van 
Keulen et al., 2018). They found that the items on the list of Hoaxmelding 
involved examples of unfounded warnings (65), polarizing disinformation 
(32) and fake crime news (31). Additionally, there were several examples 
of clickbait, benign as well as malicious. The content steers users to 
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Figure1.2 ‘Detected and eliminated’ fake news, with a warning issued by NU.nl 
and Nieuwscheckers 


Openbare berichten 


f B SMAKELIJK se Pagina leuk vinden 


‘ 


Dit is wel een wonder.. Had nooit gedacht dat dit mogelijk was 


VIDEO: VROUW van 300 KILO bevalt 
; van een BABY van 20 KILO! 
WERELDRECORD! 


Al In twijfel getrokken door NU.nl en Nieuwscheckers 


Vis B- 
de Leuk MW Opmerking plaatsen Æ Delen 


Source: NOS (2017a) 


advertising, ‘like-farming’ and phishing sites (van Keulen et al., 2018: 38). 
Such posts contain human interest stories that are ‘painful on a personal 
level’ (van Keulen et al., 2018: 45). The researchers found that only 25% of 
the disinformation concerned political content and most clickbait serves a 
commercial goal, rather than a political one. On the list of items collected by 
Nieuwscheckers, the Leiden University-based initiative, less than half was 
found to have political content. Within the full set, the researchers found 
six examples of polarizing content. Otherwise, many of the posts concern 
factually incorrect, public statements by politicians, the investigation of 
which is how fact-checking is conventionally practiced. 

Fact-checking now extends well beyond unpacking politicians’ statements, 
and Facebook has entered into partnerships with many bureaus around 
the world, including in the Netherlands, to explore and catalogue dubious 
content. In 2017 Nieuwscheckers partnered with Facebook and NU.nl and 
celebrated their first collaborative, ‘successful detection and elimination 
of fake news’ that year when they flagged a tabloid-style, human-interest 
post about an Australian new-born weighing 20 kilograms (see Figure 1.2). 
In February of 2019, however, Nieuwscheckers withdrew from the Facebook 
fact-checking initiative because of liability risks (Kist, 2019). Nu.nl continued 
to work with Facebook on fact-checking, on a paid basis, an issue raised 
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repeatedly in the context of journalists’ being asked to address an issue of 
Facebook’s making on a voluntary basis. 

The effectiveness of fact-checking as a strategy in the Netherlands is a 
different question. As mentioned above, fact-checks and fake news often 
have separate publics, and fact-checks may lead people to fake news, rather 
than away from it. A recent study in the Netherlands found that even when 
many people would agree with a fact-check, they are not interested in reading 
the fact-checking article, prompting the scholars to advise journalists to 
make the fact checks an engaging read (Hameleers and van der Meer, 2019). 
Another strategy to counter disinformation concerns a strand of media 
literacy that involves developing skills to recognize fake user accounts and 
disinformation. One is ona source level, the other on a story level. The Field 
Guide to Fake News provides a method for the detection of trolling accounts 
by looking at their friends, or their profile information (Bounegru et al., 2018). 
There are also courses and training modules for fake news detection and 
fact-checking, such as those given by Radio Netherlands (RNTC, 2019). The 
other format is the fake news quiz, such as those by de Volkskrant (2016) and 
the Guardian (2016), as well as the New York Times ‘deceptive Facebook post’ 
test (2018). These quizzes make it clear how challenging it is to recognize 
fake news. The Dutch serious game, titled Slecht Nieuws (‘Bad News’), invites 
players to create fake news and by doing so gain insight into the strategies 
behind it and become more astute in its recognition (NRC, 2018; DROG, 
2018). It is part of efforts that study false news as risk and ultimately seek 
to inoculate populations against it (Roozenbeek and van der Linden, 2018). 


Voting aid applications 


Voting aid applications (VAAs), often called stemwijzers in Dutch, are 
generally websites that describe their purpose as helping undecided voters 
find the political party that best matches their preferences and positions. 
As such, in the context of the study of disinformation and so-called fake 
news, they could be regarded as a competing persuasion instrument, a 
pre-emptive measure against influence campaigning, or even a potential 
site that may include it, either through parody, hoax or hack. The literature 
on VAAs takes up the Dutch and Belgian cases, countries that together with 
Germany, Austria and Switzerland have upwards of half the voter population 
accessing them prior to elections. The work can be positioned broadly as 
pertaining to ‘the impact of internet-based applications on politics’ and can 
be roughly divided into user studies, impacts of VAAs on the voters as well 
as the methods behind them (Hirzalla and van Zoonen, 2015: 88). To date 
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these online voting aids have not been raised as recommended technology 
to combat disinformation and influence campaigning per se, though they do 
furnish a rather personalised information experience that may be studied 
for its ‘influence’ effects, as discussed briefly below. 

In studies of their usage, researchers have asked whether VAAs ‘mobilize 
the mobilized’ (Hirzalla and van Zoonen, 2015). And indeed, while VAAs 
have a heterogenous user base across demographics, interests, attitudes and 
behaviour (Vassil, 2011), there is an overrepresented subgroup of younger, 
mainly left-of-centre, urban and well-educated male users who are politically 
active or knowledgeable. This imbalance could lead to the conclusion that 
those who may benefit from political advice are not seeking it (Ruusuvirta, 
2010). 

A second set of literature concerns the impact of VAAs and assesses 
whether they have influenced the voting behaviour of its users, though it 
is not clear whether the quality, reach and graphical interfaces of the aids 
affected the extent of the influence. From those surveyed anywhere from 
1% to 15% using DoeDeStemTest (in Belgium) as well as StemWijzer and 
Kieskompas (in the Netherlands) reported having been influenced by the 
aids (Walgrave et al., 2009; Hirzalla and van Zoonen, 2015). While research 
has found that the politically knowledgeable and engaged users that are 
common to use VAAs perceive them as useful, they are also among the 
less likely users to be influenced by them (Alvarez et al., 2014; Dumont 
and Kies, 2012). 

A third set of literature concerns the methods used by the VAAs. Here 
there is a distinction between the choice of the policy positions to include 
in the interactive system and the models underlying the advice. The very 
selection of the policy positions is a crucial factor in the voting advice 
given, where another set would lead to other advice (Walgrave et al, 2009). 
In general, VAAs are found to select policy positions according to their 
saliency (for the election period), and variability (in that different parties 
hold different positions) (Hirzalla and van Zoonen, 2015). The editorial 
process differs, where certain VAAs select their statements solely with 
experts such as political scientists or journalists (e.g., the Austrian VAA 
wahlkabine.at), while others co-create the formulation of VAA positions, 
workshopping them with party representatives in the case of the Dutch 
StemWijzer, or with an editorial board that consists of professional experts 
as well as first and second-time voters in the German ‘Wahl-O-Mat’ (Garzia 
and Marshall, 2017). 

As the voters register their political views, and in certain cases add weight 
to them, the software calculates the extent to which the voters’ preferences 
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match the respective parties' and presents its results as a ranked list, bar 
chart, grid or radar chart. Several studies concentrate on the workings and 
visual outputs of the different voting aids. Louwerse and Rosema (2o14) 
dissect them by examining how many dimensions are taken into account 
when ranking the political parties. In their study,a one-dimensionalmodel 
refers to the ranking the political parties based on the level of agreement 
with the voter and presents its findings as a ranked list or bar chart. A two- 
dimensional model places the political parties’ statements and the voters’ 
responses on a continuum from left-wing to right-wing and proposes its 
match accordingly. The more elaborate multi-dimensional model, employed 
by the Swiss smartvote application, plots the statements and responses 
onto eight policy dimensions and presents its results in a spider plot that is 
more complex to read (Louwerse and Rosema, 2014). In a comparative test 
of these models, researchers took a dataset from the Dutch Stemwijzer and 
found that the different spatial models would lead to very different matches 
(Louwerse and Rosema, 2014). 

As mentioned above, the voting aids are rather popular in a series of 
European countries and could be considered not only as another information 
input but also as one that competes with campaigning. Though the influence 
(similar to campaigning) may again be minimal, it could be considered as 
another approach or countermeasure in the discussion of how to address 
the disinformation problem. 


Junk news may be pervasive, but is it persuasive? 


If one were to divide the current period of junk news studies into waves, it 
could be argued that the first related to the definitional issues and the pro- 
duction side (as mainly discussed above), whilst the second is increasingly 
concerned with the study of its consumption (Boczkowski, 2016). In other 
words, junk news may be seen as ‘pervasive, but is it persuasive?’ (Shaw, 
1979). Why do people consume it, and do these readers have particular 
demographics or profiles? Which people deem these stories credible or 
at least have pass-along value? Are they persuaded or even persuadable? 
In the US and in a growing list of other countries social media platforms 
are increasingly a main source of news, and the manner in which they 
deliver news is different from a newspaper or similar package or container 
(Gottfried and Shearer, 2016; Poynter, 2019). One receives single stories, 
rather than an entire newspaper, each shared by someone with whom the 
social media user has made a connection, most often directly. These can 
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be friends (Facebook), followers (Twitter), connections (LinkedIn), etc. 
Stories arrive in the feeds algorithmically, meaning there is a filtering 
mechanism where certain of them are boosted, based on signals such as 
activity and increasingly trustworthiness, or the amount of given and 
measured meaningful engagement between individuals. Put differently, 
those who are close to the user (by some special measure) are the ones 
whose stories more likely will be seen (Eslami et al., 2015). Such observations 
have led to discussions of the re-application of the notion of the filter 
bubble, a term originally associated with a user receiving personalised 
(rather than universal) search engine results (Pariser, 2011; van Keulen et 
al., 2018; Puschmann, 2018). Personalisation, however, has evolved from 
being the result of the information interactions of one user searching to 
engagement with an entire social network. As such it shifts the bubble 
from enveloping the individual to the group; it has prompted ‘bubble 
studies’ of not just social media news environments, but those of health, 
science, fashion and other areas of collective information production, 
sharing and recommendation (Pedersen and Hendricks, 2014; Hendricks 
and Vestergaard, 2019). Indeed, junk news circulation and consumption 
are increasingly experienced as an issue for the environment (e.g., climate 
change and its sceptics), health (e.g., the anti-vaccination discourse) and 
a variety of other areas (Kitta, 2018). 

Such findings have led researchers to define on the one hand the groups 
most likely to consume and share the news together with the dynamics 
of their bubbles, and on the other the meaning, or sincerity, attached to 
the sharing. In terms of the consumption of junk news, it could be said 
at the outset that there have been two widely cited findings about their 
significance from the journalistic arena. One found the most shared stories 
during the US presidential elections were ‘fake news’ (see Figure 1.3), and the 
other that Russian disinformation campaigns had a far greater spread than 
previously imagined as well as reported in testimony by Facebook before the 
US Congress (Silverman, 2016; Timberg, 2017). These findings have since been 
put into a broader context and compared to ‘normal’ political campaigning 
and the development of messaging strategies, filtered through news. First, 
in the event, only a small fraction of the population consumed such ‘news’ 
(Allcott and Gentzkow, 2017). Given the limited exposure, the impact, if at 
all, would have paled in comparison to political TV commercials (Persily, 
2017). There is the larger question, however, of whether the messaging would 
have anything but ‘minimal effects’ (Lazarsfeld et al., 1948). As has been 
repeatedly found, the net effect of campaigning, albeit by political elites, 
that persuades the prospective voter is exceedingly low or even zero (Kalla 
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Figure 1.3 The birth of the fake news crisis, or “fake news’ outperforms ‘mainstream 
news’ on Facebook, in the run-up to the U.S. elections in 2016 
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Source: Silverman (2016) 


and Broockman, 2017). The aim then is less to persuade than to ‘rous[e] the 
enthusiasm of existing supporters’ (Panagopoulos, 2016). 

Though they may have begun as symmetrical studies of the right and 
the left, of the most circulated findings to date about the spread of junk 
news — both with respect to the contents as well as its consumers — ultimately 
all have overwhelmingly concentrated on the right, be it conservatives 
and the alt right in the US or other right-leaning, populist radical right or 
new right publics in Europe (Bounegru et al., 2018; Benkler et al., 2018). 
It is of interest to note for starters that both during the US presidential 
campaigning and thereafter the information spaces or spheres of the right 
contained far more fake, junk, disinformation or otherwise dubious stories 
and sources than the left (Faris et al., 2017). Thus, conclusions drawn about 
right-leaning publics sharing information should take into account that they 
are disproportionately exposed to such information; all else being equal, the 
right would share more of it (Marwick, 2018). In the empirical studies it was 
found that the right (most notably Trump supporters) consumed the most 
so-called fake news. However, there seems to be an older, hard core of its 
consumers in the US during the run-up to the US presidential elections in 
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2016 — ‘the 10% of Americans with the most conservative information diets’ 
(Guess et al., 2018: 11). These are heavy media users, and ‘available audiences’, 
who have made time to consume media (Nelson and Taneja, 2018). Unlike 
the majority of the media-consuming public, they are far more likely to read 
niche rather than only establishment sources. There is, in other words, a 
normalcy to the consumption by those audiences of fringe materials. 

The strand of work that considers why users share ‘fake news’ should be 
prefaced by the distinction between ‘earnest and ambivalent’ internet users 
(Hedrick et al., 2018). Much of the scholarship about internet culture has not 
considered that considerable cultural production and sharing are undertaken 
not to be part of participatory culture, connective action and other earnest 
forms of civic culture online but rather for unsympathetic amusement (aka 
‘lulz’) (Phillips, 2015). ‘Sharing’, a term that has mutated in digital culture 
from acting in a gift economy to a dominant form of so-called platform 
capitalism, could have been prompted these days as much by insincerity 
as by mindfulness (Barbrook, 1998; Belk, 2007; Srnicek, 2017). That is, the 
rationale for making and sharing could ‘go either way [...] complicating an 
easy assessment of authorial intent’ (Phillips and Milner, 2018: 10-11). Such 
a tricky attribution of intent is especially troublesome in the spaces where 
vitriolic exchange as well as extreme speech and content are prevalent. It 
is difficult to disentangle whether one is sharing for amusement and to 
trigger a reaction, or for substantive reasons. 

As has been found in the US context, the problematic news stories most 
shared on social media resonate with particular grievances (about the bias of 
establishment sources) and resentments (concerning economic opportunity) 
that underlie certain societal divides (Marwick, 2018). Moreover, the stories 
do not stand alone in a mirror world of conspiracy theory but rather are 
contiguous with more mainstream conservative news, anchored by Fox 
News; they are more extreme as well as transgressive in their wording and 
presentation. Hence the notion of ‘hyperpartisan’, but there is also reference 
made to tendentious, anti-establishment sources. Here the Overton Window 
is appropriately referenced, meaning the bounds of current, acceptable public 
discourse, and the extent to which extreme speech in hyperpartisan and 
tendentious sources is moving established norms (Daniels, 2018). 


Junk news studies: Digital methods and data journalism 


As we come to shortly, one research strategy for measuring the prevalence 
of problematic news story types and sources around national elections 
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is to gauge their presence generally in scoping exercises, but also more 
specifically in the most engaged-with content in social media concerning 
elections, political parties, candidates and social issues. A more subtle 
analysis would examine the top stories for the penetration of problematic 
news narratives, measuring mainstreaming. Moreover, through comparison 
of engagement with such news, one also could determine which platforms 
are most susceptible (or amenable) to hosting and circulating such content. 
Facebook in particular has been held up as a ‘hyperpartisan media machine’ 
(Herrman, 2016). Empirically, it has been found to host (proportionately) 
more of it, whether narrowly or liberally defined, than other platforms 
(Guess et al., 2018). 

In order to contextualize such measures, it is worthwhile to consider the 
ways in which the scale, reach and impact of such news have been studied 
to date with media analysis, or more specifically digital methods and data 
journalism. The methods generally could be considered mixed quantitative/ 
qualitative approaches. They often begin in the journalistic arena, with 
the qualitative determination of the dubiousness of a set of sources and/or 
stories and proceed with digital methods that design queries and collect 
data from platform APIs, media monitoring company dashboards, and 
social media companies that have furnished lists of banned trolls or user 
accounts. Indeed, with respect to the dubious source lists, Buzzfeed News’ 
original list of about 20 sources determined to be ‘fake news’ inform a 
series of empirical studies (Silverman, 2016; Allcott and Gentzkow, 2017; 
Bounegru et al., 2018; Marwick, 2018; Grinberg et al., 2019). For studies of 
the Italian news space, the lists relied upon are from BUTAC, Bufale and 
Bufalopedia (Fletcher et al., 2018; Butac, 2018). Hoaxwijzer’s list of 92 Dutch- 
language ‘false news’ sites also informs certain of the empirical studies to 
date in the Netherlands (van Keulen et al., 2018; Wieringa, 2017).* But other 
work, such as the NRC Handelblad’s analysis of the extent of the problem of 
‘fake news’ in the Netherlands in the run-up to the 2017 national elections, 
looks at the sources afresh, making on-the-spot determinations of fakeness 
(Kist and Zantingh, 2017). These may conflict with previous listings. For 
example, Hoaxwijzer lists De Dagelijkse Standaard as a ‘false news’ site 
whereas the NRC Handelsblad did not determine it to be ‘fake news’, but 
it fell among those they called ‘misleading’ because it reported that ‘1,000 
crazy Muslims’ had ‘torched’ a church in Dortmund on New Year's Eve when 
instead a firework had landed on its roof causing light damage. The NRC 


4 As other studies also found, the list is dated; as of April 2019, 40 of the 92 sites are offline. 
It remains useful as a list for older media corpora. 
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Handelsblad determined that it did not meet its definition of fake news as 
a “fully fabricated story packaged as news’. 

Indeed, the question of detecting fabricated news, on a source or story 
level, is often placed at the feet of journalists, media organizations and 
fact-checking bureaus, where credibility and transparency may be rated 
(NewsGuard, 2019). Masked sources are penalised, for example. As mentioned 
above, for online stories, the determination of dubious content may benefit, 
too, from a genre analysis (Liiders et al., 2010). Disinformation, conspiracy, 
clickbait and (automated) amplification have styles (Rony et al., 2017). 
Disinformation tends to be a hard counterfactual presentation, conspiracy 
has multiple characters and plot entanglements, clickbait is a cliff-hanger 
that is often painful on a personal level and (automated) amplification 
posts at particular intervals and in coordination, as malicious social bot 
detection projects have found (Ratkiewicz et al., 2011; Bessi and Ferrara, 
2016; RoBhat labs, 2017). Other technical signatures of dubious news sites 
are of interest. For instance, empirical work on the types of cookies and 
third-party elements in mainstream and nominally fake news sites found 
distinctive types in each, with the mainstream sites using customised 
trackers and the other off-the-shelf (Bounegru et al., 2018). 

With the lists of fake news sources either in place or determinations still 
to be made, the next step is to build a media corpus. Following Buzzfeed 
News’ method, many undertakings query media monitoring services (such 
as Buzzsumo and Facebook’s Crowdtangle) for political and issue-related 
keywords, in order to build source sets of most engaged-with media and 
pull in engagement scores per story. Certain of the techniques also include 
further interpretative coding of stories, including grievance narratives 
(Marwick, 2018). 

Whilst much attention has been directed towards Facebook, and the 
study of the election-related stories most engaged with on that platform, 
Twitter is often used as the preferred data source, given dedicated data 
sets (made available by Twitter or academic researchers) of accounts run 
by the Internet Research Agency (Farkas and Bastos, 2018). There is a series 
of studies that rely on Twitter's curated sets as well as on the data robustly 
collected and shared among data researchers, such as by Clemson University 
and FiveThirtyEight, mentioned above. In a form of crowd science, the 
publication on GitHub of the Clemson data set led to numerous studies; in 
the US widespread disinformation campaigning was found, as is known, 
but also more niche-targeting of politicians in such states as Maine (Roeder, 
2018). As in the Netherlands, discussed above, the data were put to use in 
other countries that according to journalistic accounts had been previously 
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understudied. For example, in Italy IRA trolls posted numerous pro-populist 
party tweets in Italian, joining the ‘cacophony’ or media ecology around the 
populist right, as discussed in the Oxford Internet Institute work (Fubini, 
2018; Fletcher et al., 2018). Twitter is also preferred given the general ease of 
use of data collection through its streaming and search APIs, intermediate 
services such as Hexagon Crimson for samples as well as the availability of 
historical data sets, albeit at a price. 

Other approaches (considering consumption and persuasion rather than 
definition and production) should be touched on that rely on surveys, 
user data collection and experiments. Numerous experiments have been 
performed on misinformation (Jankowski, 2018). For example, a repre- 
sentative sample of the population consents to having their online media 
consumption passively monitored, and subsequently surveyed thereafter 
(Guess et al., 2018). Or, there are experiments that show fabricated news 
to consumers, and subsequently provide fact-checks to them in order to 
determine whether the fact-checks should be ‘attitudinally congruent’ for 
them to be persuasive (Hameleers and van der Meer, 2019). In another experi- 
ment in the Netherlands, commissioned by the newspaper, de Volkskrant, 
respondents were presented with fabricated news around one of four topics: 
vaccinations and autism, MH17, rape incidents in connection to migration, 
or Sylvana Simons (a politician and media personality) and discrimination. 
The study tests whether they became less certain about the facts after being 
misinformed (I&O Research, 2017; Kranenberg, 2017). 

With respect to platforms other than Facebook and Twitter, YouTube and 
especially Instagram appear to be relatively understudied but significant, 
and Reddit and 4chan are being recognized as breeding grounds for some 
of the more outlandish and consequential content such as Pizzagate (New 
Knowledge, 2018; Tuters et al., 2018). There are platform-specific approaches 
for building and analysing datasets for Instagram (through queries for 
hashtags and place names), as well as YouTube, Reddit and 4chan (Rogers, 
2018b; Rieder et al., 2018). Google web search also has invited scrutiny, given 
the extreme content returned for queries such as the Holocaust. 

Buzzfeed News’ work on detecting and analysing ‘fake news’ on Facebook 
has been particularly influential in data journalism research and subsequent 
studies that build upon it, and thus is worthy of mention in some detail 
(Silverman, 2016). First, the researchers built a keyword list concerning elec- 
tions (and especially controversial election topics), and subsequently queried 
those keywords in media monitoring software (Buzzsumo) that returns 
stories ranked by engagement scores. With the aid of the results, they built 
a fake news and hyperpartisan website list, which they merged with lists of 
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the same that they curated previously through separate reporting, including 
on the infamous collection of about 100 websites created by the Macedonian 
clickbait makers, members of the same family of sites (with the same Google 
Analytics ID) of WTOE 5 News that created the story about the Pope endorsing 
Trump, and a collection of hyperpartisan sites (Silverman and Singer-Vine, 
2016; Silverman et al., 2016). They also curated a list of some 20 mainstream 
news sites 3 (All the accompanying data Buzzfeed also made available through 
online Google spreadsheets, in keeping with emerging standards in data 
journalism.) The engagement scores of the top mainstream news and top 
fake news stories are subsequently compared. In the first study of this kind 
and perhaps the beginning of what could be called the ‘fake news crisis’ for 
Facebook, it was found that the fake news stories outperformed by engage- 
ment scores those from the mainstream news in the three-month period 
before the US presidential elections, thereby leading to conclusions about 
the comparable ‘power of fake election news on Facebook’ (see Figure 1.3) 
(Silverman, 2016). Follow-up reporting has considered the extent to which 
such news continues to resonate more on Facebook than mainstream news 
stories, despite incipient efforts by the company to curtail its impact. One of 
the major studies commissioned by the US Congress found that such news 
and influence campaigning activity on Facebook and especially Instagram 
substantially increased after the US elections (Howard et al., 2018). 

In April of 2019, some two and one-half years after Buzzfeed News story, 
we found that only 4 of the 13 top-performing ‘fake news’ and hyperpartisan 
websites are still online: World News Daily Report, Burrard Street Journal, 
Twitchy and Breitbart. The others appeared to have been fly-by-night opera- 
tions, which is another means of considering a source’s dubiousness. That is, 
the other o sites, including two Macedonian-made ones (Denver Guardian 
and World Politicus) and the highest-performing site (Ending the Fed) that 
spread the ‘Pope endorses Trump’ story are gone. 


Facebook’s adjustments 


After the US elections in 2016, Facebook CEO Mark Zuckerberg initially 
argued that ‘the idea that fake news on Facebook influenced the election in 


5  Buzzfeed’s list contains the following mainstream sources: New York Times, Washington 
Post, NBC News, USA Today, Politico, CNN, Wall Street Journal, CBS News, ABC News, New York 
Daily News, New York Post, BuzzFeed, Los Angeles Times, NPR, The Guardian, Vox, Business 
Insider, Huffington Post and Fox News (Silverman, 2016). 
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any way, I think is a pretty crazy idea, and put forward that such material 
amounted to a small fraction of its platform's content (Isaac, 2016). Two years 
later Facebook’s work to quell fake news, together with its more stringent 
policies on (research) data access should be considered here. Addressing 
the fake news crisis, there has been an increase in those hired to perform 
‘content moderation’, referred to as the janitors of social media, or even those 
doing the platform's dirty work (Roberts, 2017). Facebook also installed a 
political ad transparency tool; it lists on the ad itself who has sponsored 
it, and there also is a political ad archive and an API (Hern and Waterson, 
2018). No longer is the maker and targeted individual the only ones able to 
view the hitherto ‘dark post’. 

With respect to algorithmic changes, in 2018 Facebook began a three- 
pronged strategy that would favour ‘meaningful connections’ (family and 
friends), ‘trusted sources’ (user-surveyed media) and ‘local news’ in the news 
feed over more far-flung ‘businesses, brands, and media’ (Abbruzzese, 2018; 
Flynn, 2018; Gartenberg, 2018). It should be remarked that these are global 
initiatives, coming on the heels of well-reported Facebook-associated riots 
in Myanmar and Sri Lanka but also the compilation of compendiums on 
the effects of ‘fake news’ on Facebook all over the world, as the OIT's global 
study have shown, but also the numerous governmental and think tank 
(umbrella) initiatives such as disinfoportal.org. 

Whether Facebook’s measures are working in some sense is unclear. The 
political ad library tool may show a source, but who is behind it may remain 
unclear as in the case of a pro-Brexit campaign group, Britain's Future (see 
Figure 1.4), that spent hundreds of thousands of pounds on ads in the run-up 
to significant UK parliamentary votes (Waterson and Hern, 2019). Efforts 
by journalists to unmask the source behind Britain’s Future as well as other 
‘dark money’ campaigners had for months been in vain (Monbiot, 2019). 
Significant political ads are also not in the archive, as ProPublica found, 
before its tool crowdsourcing Facebook ads and targeted individuals was 
purposively rendered inoperable by the company in what it called a ‘routine 
update’ that would prevent illegitimate ‘scraping’ (Merrill and Tobin, 2019). 
Similar tools by Mozilla and Who Targets Me also broke, thus making the 
verification work a difficult prospect. 

The news feed tweak to boost ‘meaningful connections’ was initially 
critiqued for its capacity to exaggerate the importance of ‘fake news’, as 
was observed in Slovakia and elsewhere when dubious sources saw their 
engagement scores rise (Frenkel et al., 2018). The prominence of ‘meaningful 
connections’ and ‘local news’ in the news feed, according to Buzzfeed, stirred 
as well as amplified the Gilets Jaunes protests in France, for their coverage 
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Figure1.4 Facebook political ad library tool, results for Britain’s Future, 


13 March 2019 


Fimer resum by 1300 rests 


maen 


PILL D&D 10 DELATE wt r 


on the local news made the anger groups (groupes colëre) and their posts 
more prominent in the news feeds, as evidenced by engagement scores from 
Crowdtangle (Broderick and Darmanin, 2o18). 

Given the fake news crisis stemmed from the US elections, Facebook 
also created specific initiatives for future elections that would put political 
parties and their positions on issues in a single, curated Facebook portal. 
One of the early projects was for Sweden's national elections in 2o18, which, 
it was found in a separate study (with Twitter data), suffered from ‘junk news’ 
quantities second in magnitude only to that surrounding the US elections, 
and much larger in fact than such materials around the German, French 
and Dutch elections in 2017 (Hedman et al., 2018; Kist and Zantingh, 2017). 
The Facebook elections project, rolled out in meetings with social media 
researchers in 2018, also coincided with their new academic ‘partnership’ 
project, Social Science One. It seeks to make available to researchers data sets 
such as all the URLs that have been posted to Facebook over the course of a 
year (King and Persily, 2018). At the same time, however, Facebook revoked 
approval for research software (such as Netvizz and Netlytic) that made use 
of its Pages API, sparking academic protest about ‘locked platforms’ (Bruns 
et al., 2018; Rieder, 2018). Seen as reactions to the Cambridge Analytica 
scandal, Facebook’s measures could be described as curating the datasets 
researchers can use. The new datasets (that would be available in the Social 
Science One initiative) notably do not include Facebook pages themselves 
and their engagement scores — data that led to the very knowledge about 
the fake news crisis and the scope of the Russian influence campaign in 
the first instance (Albright, 2017). 
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Conclusions: Fakery and campaigning 


The ‘fake news’ phenomenon could be viewed as a revival of previous ones 
that typically have occurred when a new media technology is introduced that 
destabilises production, distribution and consumption of news and informa- 
tion, as was the case with eighteenth and nineteenth century broadsheets 
and tabloids (respectively) but also the radio and newswire of the twentieth 
century. The early web and the blogosphere also challenged existing news 
publication practices and were considered unedited spaces populated by 
self-styled authors, providing speedy news ‘too fresh to be true’. Now social 
media platforms disrupt the trustworthiness of established news and fact 
and reintroduce the idea of the web as ‘truthless medium’ (Marres, 2018). 

The post-truth age, or condition, as it were, may be viewed in light of a 
conflict between what counts as ‘fake’ (on a source or a story level), but it has 
been described rather as a contest between facticity regimes, or even sets 
of sectarian expertise. Locating a network of so-called ‘fake news’ websites, 
for example, could be viewed as the discovery of an influence campaign, 
but it just as well can be seen as an ‘alternative facts’ media ecology. When 
it is a hyperpartisan, right-wing news ecology, as in the US in the run-up 
to the presidential elections of 2016, it could be described as a part of the 
contemporary post-truth situation, or, as been often related, a culture war. 

Having the ‘trappings of news’ in terms of look and feel, ‘fake news’ has 
been defined as consisting of distinctive types with varying intentionality. 
For instance, disinformation and mal-information (the neologism) are meant 
to harm, whereas misinformation may be just as false but its circulation 
unintentional. As a case in point, satirical stories and parody may become 
misinformation, such as the story about the Pope endorsing Trump, which 
outperformed (by engagement score) any other ‘news’ on Facebook during 
the US presidential election campaigning in 2016. 

In both the public as well as scholarly discourses, there has been a swing 
from the hype of the ‘fake news’ problem (perhaps well exemplified by the 
Dutch tabloid headline ‘landslide of fake news’) to its gradual debunking, 
e.g., ‘researchers say fears about “fake news” are exaggerated’ (Ingram, 2019). 
Such a view has resulted from a series of studies not just on engagement but 
also on its consumption, including the rationale behind its sharing. Small, 
older populations appear particularly active, as do ‘heavy news consumers’ 
and ‘available audiences’, or those who have at their disposal time for fringe 
news consumption and spreading it among online friends. The vast majority 
of news consumption remains of the mainstream sources, however. The 
evidence that consumers have been influenced or persuaded is minimal. 
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Nevertheless, there appears to be agreement that social media platforms 
remain worthy of study not only as the new ‘truthless medium’ but for their 
capacity to accelerate (or ‘supercharge’) ‘fake news’ distribution in a hybrid 
media system comprised of new and established media and media formats. 
Despite increased content moderation, automated detection work, and a 
reorientation of its news feed principles, does Facebook remain a ‘fake news’ 
machine, comparable to the one during the US presidential campaigning? 
Indeed, Facebook, at first hesitant to admit an issue, has taken a series of 
measures since then that strive to produce more trustworthiness, such as 
boosting posts by friends and family, crowdsourcing trusted sources as well 
as favouring local news, though the effectiveness of these reengineered 
principles has been questioned. Indeed, continuing empirical research on 
the most engaged-with, political news on Facebook could shed light on the 
quality of the platform’s content delivery, however much data access may be 
restricted to researchers. It remains to be seen how ‘oversight’ research will 
be affected now that Facebook has closed research APIs and instead plans 
to curate data sets for researchers, rather than allowing them to create their 
own. Other oversight projects have been thwarted; in early 2019 Facebook’s 
‘routine update’ blocked the software by ProPublica, Mozilla and Who 
Targets Me that was collecting political ads and their targets, as mentioned. 

The question of ‘fake news’ as a campaign strategy — be it by Russian 
operatives, Russified domestic actors, hyperpartisan media-makers, and 
others — also has been meticulously studied, with detailed ‘playbooks’ 
laid bare as tactics to create both a media mirage (where fact and fiction 
are difficult to disentangle) as well as competing truth regimes, offering 
counter-expertise as well as uncertainty. Governments around the world 
have commissioned studies, revealing the breadth and scope of the problem, 
explaining the playbook and putting forward policy recommendations 
such as increased media literacy and the regulation of political advertising 
on platforms, including ‘dark’ posts. Platforms are asked to create public 
archives, which also would benefit research as well as (data) journalism. 
Fact-checking also has gone global, though it often remains a small-scale 
enterprise practiced by bespoke bureaus, occasionally working in tandem 
with Facebook, checking posts that have been flagged by users, and weighing 
in on the question of fakery. 

Finally, there are scholars in the US and recently in Europe putting 
forward the argument that studying Russian disinformation shifts the 
attention away from the home-grown hyperpartisan news ecologies that 
have been emerging over the past few years, particularly on the right 
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(Benkler et al., 2017; Benkler et al., 2018; Rone, 2019). The point also fits 
with the ‘dark globalization’ argument concerning how existing domestic 
divisions, displayed in this media, may be exacerbated by foreign operatives 
but are not created by them. To date the effectiveness of Russian influence 
campaigning in Europe, in either sowing or exacerbating division, has yet 
to be compellingly demonstrated; the false and junk domestic news sources 
(e.g., the pro-Russian sources re-narrating the cause of the downing of 
Miz) also appear to have scant reach (Fletcher et al., 2018). In a climate of 
heightened sensitivity towards dubious sources and stories, it remains to 
be seen whether they have staying power. 


KEES 


Appendix: Governmental efforts and discussions of 
countermeasures 


A first step for many national governments and other regional political 
entities that wish to counter disinformation is to install committees as well 
as task forces; it occurs across the globe, from the much publicised hearings 
by the US Congress and UK Parliament on the Russian involvement in the 
US elections and the Cambridge Analytica affair, to the task forces and 
other entities formed in many of the nearly 50 countries where influence 
campaigning has taken place (Bradshaw and Howard, 2018). Following from 
these convenings, there have been national calls to regulate the ‘digital 
giants’, and the European Union, through its creation of a High-Level Expert 
Group (EU HLEG) on fake news and online disinformation, has issued 
its recommendations for countering disinformation, including calls for 
transparency, media and information literacy, and tools for empowering 
journalism. In the European countries with recent or imminent national 
elections there has been even greater urgency, with Germany and France 
enacting legislation (online hate speech and ‘fake news laws’, respectively), 
and Sweden and Denmark engaging in awareness-raising as well as media 
literacy campaigns. Denmark installed a ‘digital ambassador’ (Gramer, 
2017). 

Below is a list of certain measures to counteract disinformation and fake 
news, gleaned from governmental documents and related materials. They 
include social media company regulation, codes of ethics, fact-checking 
and media literacy campaigning. 
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Social media company regulation 


Many government committees agree that the large tech companies that have 
come to dominate the online realm, such as Google, Twitter, and Facebook, 
should be regulated, but caution over-regulation in forms that would curtail 
expression and press freedoms. The starting point for the regulation ofthese 
companies to counter disinformation is to address political advertising 
on social media platforms. It can include the verification of those paying 
for political advertisements and disclosing them publicly. Additionally, 
all social media companies could be required to create public archives of 
advertisements so that among other ad types ‘dark posts’ may be studied 
(Bradshaw, 2018). In fact, as said, Facebook has such an archive (and an 
API), but it also prevented watchdogs including Mozilla from verifying 
its collection techniques, equating their methods with illegitimate data 
‘scraping’ (Merrill and Tobin, 2019). 

Relatedly, the EU HLEG proposes the development of a ‘European-wide 
code of practices’ that describes the roles and responsibilities of relevant 
stakeholders such as tech companies, and media organizations but also 
research organizations and fact-checking initiatives, based on key principles 
(2018). In short, they address the adaptation of political advertising policies 
(including sponsored advertisements and other forms of content), and 
the provision of access to data for research and fact-checking. They also 
propose the installation of advanced settings for users to customise their 
user experience, collaboration with news outlets to facilitate users’ access 
to trustworthy news, the facilitation of fact-checking and content flagging, 
and allowing users to ‘exercise their right to reply’ (EU HLEG, 2018: 32-33). 

The UK Parliamentary report on fake news and disinformation speaks 
in an unusually piqued tone of the importance of regulating social media 
platforms and related tech companies, singling out Facebook as providing 
the ‘impression of working towards transparency’, but often ‘obfuscating’ 
how well it is capturing and archiving political ads (House of Commons, 
2019: 85). Ultimately, they propose the establishment of an ‘educational 
levy’ or charge on social media companies to fund digital literacy as a 
fourth pillar of the education system after reading, writing and maths 
(House of Commons, 2019: 87). There is also a recommendation that social 
media companies should develop means to distinguish between those 
sources regularly furnishing disinformation and those who do not, in a new 
system of ‘content regulation’ (House of Commons, 2019: 87). While carefully 
worded, that measures can count on the criticism that similar proposals 
have faced concerning the restriction of the freedom of expression, while 
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not being effective measures against hateful or incendiary content (Access 
Now et al., 2o18). 

Nevertheless, legislation has been passed. Germany has established a 
law, NetzDG, that extends its hate speech legislation compelling social 
media companies (with more than two million registered users in Germany) 
to remove such speech rapidly or face hefty fines (Claussen, 2018). More 
controversially, France has new legislation which applies to ‘false informa- 
tion’; the law requires that three months prior to an election ‘false news’ 
be removed. 


Detecting and removing false content 


The Reporters’ Lab at Duke University keeps track of fact-checking initiatives 
worldwide and has identified some 160 active initiatives (Duke Reporters Lab, 
2019). In European countries, some fact-checking initiatives are attached to 
news organizations, but most are operating as not-for-profits (Wardle and 
Derakhshan, 2017; Graves and Cherubini, 2016). Many work in tandem with 
Facebook; as of January 2019, some 50 fact-checking groups, who are party to 
the International Fact Checking Network Code of Principles, independently 
assess fake news flagged by users (Volpicelli, 2019). The expertise developed 
includes a variety of flagging and adjudication systems such as NewsGuard’s 
‘nutrition label’ that evaluates some 2,000 online news sources, or, as it 
relates, the sites that garner about 95% of engagement in the news sector 


(2019). 
Automation 


Brief mention should perhaps be made of automation as offering methods 
for flagging dubious or false content, however much it is rarely recom- 
mended in governmental reports. With respect to fact-checking, if there are 
shared databases of ‘already fact-checked’ stories as well as sources, then 
software could cross-check suspicious ones against those already debunked 
or evaluated, as the UK parliamentary report mentions. The discussion 
concerning the need for human reviewers for content interpretation and 
curation remains pertinent. 


Counter-narratives 


In Germany the government chooses to actively participate in spaces where 
disinformation is spread. ‘On these platforms, the German Government 
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provides both reliable information that can be fact-checked and a narrative 
based on this information’ (German Federal Foreign Office, 2018). In that 
vein, rumoursaboutgermany.info is a website for collecting and counter- 
ing disinformation about Germany spread by human traffickers. While 
Germany chooses to work with counter-narratives, others have criticised 
this approach. A Canadian intelligence report argues that developing 
counter-narratives is a ‘one event at a time approach’ that ‘fails to address 
the source and methodology of information campaigns’ (Canadian Security 
Intelligence Service, 2018: 66). 


Media literacy and digital ‘hygiene’ 


The EU high level expert group on fake news and online disinformation 
makes a case for increased media and information literacy to counter 
disinformation, which should be ‘implemented on a massive scale in school 
and teacher training curricula’ (EU HLG, 2018: 26). This media literacy 
also should involve the development of tools and training modules for 
journalists. As a particularly relevant method, the group proposes ‘more 
powerful tools to be able to visually map online networks and connections 
to understand how disinformation is being created, spread and amplified’ 
(EU HLG, 2018: 28). 

Some countries speak of ‘digital hygiene’ when referring to media literacy 
practices, for instance in France when making a case for the develop- 
ment of skills to assess the validity of the arguments and the reliability 
of the source. ‘This is a public hygiene measure - just as people in the 
igth century learned to wash their hands’ (Jeangène Vilmer et al., 2018: 
179). In Sweden the word ‘cyberhygien’ is employed. The Swedish Civil 
Contingencies Agency has published a handbook for communicators in 
public sector organizations for the countering of disinformation, which 
includes strategies that range from source checking and recognizing a 
bot to choosing an appropriate response to disinformation. The Swedish 
Media Council developed a media literacy programme for young people, 
teaching them critical thinking and disinformation detection; it includes a 
set of educational materials on ‘source criticism’ (‘Kallkritik’) (Government 
Offices of Sweden, 2017; Swedish Media Council, 2019). Several recent 
reports stress the importance of better equipping journalists with tools and 
skills to recognize and avoid disinformation, mentioning the importance 
of fact-checking, critical source assessment and ethics (Jeangène Vilmer 
et al., 2018; Wardle and Derakhshan, 2017). 
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Investing in civil society and building public trust 


A more general way forward that is presented in the literature is to invest 
in civil society, as it ‘must remain the first shield against information ma- 
nipulation in liberal, democratic societies’ (Jeangéne Vilmer et al., 2018: 169). 
Such initiatives are specifically relevant around events such as elections, in 
which civil society can be supported through non-legislative, pre-emptive 
measures and multi-stakeholder collaboration of government with the 
industry, non-governmental sector, and regional actors (Haciyakupoglu 
et al., 2018). In Sweden, the aforementioned Swedish Media Council is an 
example in which politicians and media professionals collaborate and meet 
regularly to discuss and counter disinformation and related challenges. Such 
regular, multi-stakeholder consultation both within and across European 
countries is among the recommendations often given (Brattberg and Mauer, 
2018). 


Guaranteeing participation in public debate by all 


Lastly is the admonition issued in the 2017 joint UN declaration on ‘fake 
news’ that emphasized the need for states to enable the participation of all 
in public debate. They should ensure that any efforts to quell or thwart the 
practices of fake news-making and spread as well as that of disinformation 
be handled within the context of the freedom of expression and the freedom 
on the press (McGonagle, 2017). 
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2 Political news on Facebook during the 
2019 Dutch elections 


Stijn Peeters and Richard Rogers’ 


Abstract 

This chapter discusses Facebook-based engagement with news sources 
during the campaigns for two Dutch election campaigns in 2019. Building 
on earlier journalistic and academic work, a broad typology of ‘junk’ versus 
mainstream news is developed, as well as a number of more specific alter- 
native categories. Engagement with news articles within these categories 
on Facebook is then analysed with BuzzSumo (a media monitoring service 
built atop CrowdTangle). While mainstream news receives significantly 
more engagement than other types of news during both campaigns, 
junk news also receives consistent and significant engagement, though 
no substantial engagement with outright disinformation is found. We 
conclude with a cursory comparison of the findings with those for other 
social media platforms, positioning Facebook as the platform where 
engagement with junk news is most significant. 


Keywords: Facebook, news engagement, junk news, cross-platform analysis 


Introduction: Facebook 


Since 2016 online disinformation and so-called fake or junk news have been 
virtually synonymous with social media platforms, serving as their most 
significant conduits. The 2016 U.S. presidential elections and the British 
Brexit referendum of the same year opened a period of increased scrutiny 
of these platforms in how false or misleading information are published 
and amplified. Facebook, the single largest social media platform of the 


1 The research reported here was undertaken in collaboration with Tim Groot. 
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past decade, has been an obvious focal point. It has been the subject of a 
substantial and growing amount of studies that investigate its ‘challenge 
[to] journalism’ (Johnson and Kelling, 2018: 817), the persuasiveness of fake 
news shared on it (Allcott and Gentzkow, 2017) and the prevalence of it in 
the average user's Facebook practice (Guess et al., 2018). 

One of the first well-publicized reports on this topic, and the one that 
informed some of the subsequent research, was BuzzFeed News’ 2016 story on 
the prevalence of ‘fake news’ in the three months leading up to the presidential 
elections that saw Donald Trump elected the 45"" president of the United 
States. The report, entitled ‘This Analysis Shows How Viral Fake Election News 
Stories Outperformed Real News On Facebook’ (Silverman 2016), outlines 
user engagement with ‘fake news’, finding that in the last few weeks before 
the election it was engaged with more often than mainstream news. 

Following this piece and other coverage on the prevalence of ‘fake news’ on 
its platform, Facebook repeatedly announced initiatives that were ostensibly 
intended to prevent it from happening again by employing third-party 
fact-checking organizations (Mosseri 20174), giving ‘more informative’ 
content higher priority (Mosseri 2017b), providing more information about 
the authors of news content (Hughes et al., 2018) and increasing content 
moderation. Despite these changes, a few years after the 2016 US elections 
the platform has still repeatedly been found in studies to be spreading 
problematic content. It has been criticized because of its role in spreading 
false and hateful content about minorities in Myanmar (Fink, 2018), live 
streaming the 2019 Christchurch mass shooting (Shead, 2019) and in inciting 
religious hatred in Bangladesh through viral content that is misleading 
(Haque et al., 2018: 1). In an analysis of social media use around the Mexican 
presidential elections in 2018, however, only ‘limited evidence of junk content 
on [Facebook]’ was found (Glowacki et al., 2018: 4). Similarly, a 2017 analysis 
of social media usage by Dutch political parties found scant ‘dubious’ content 
shared by Dutch political Facebook pages (Wieringa et al., 2017: 60), though 
their focus was Facebook pages associated with political parties rather than 
a larger Dutch Facebook sphere. 

Facebook therefore remains an interesting object of study. It is both 
the platform most commonly associated with dubious content as well as 
one that, taken at face value, has been relatively proactive in deploying 
initiatives against its spread. Additionally, existing literature is inconclusive 
with regards to the extent to which these measures have been effective, 
and there seem to be significant regional differences in the penetration of 
‘fake news’ in the discourse on the platform, and its effects. There is some 
existing research focused on the overall Dutch media sphere, most notably 
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a study on fake news during the 2017 Dutch parliamentary elections by the 
NRC Handelsblad, the national newspaper. The NRC Handelsblad found 
little evidence of the phenomenon; however, as both Dutch politics and 
Facebook’s platform have undergone changes since then, the two Dutch 
elections of 2019 — the provincial elections (provinciale statenverkiezingen) 
and the EU Parliamentary elections — present a useful case study through 
which one may investigate the extent to which disinformation and ‘fake 
news in a broader sense play a role in this particular geographical context 
on the platform, three years after the 2016 U.S. elections, and two years 
after the previous major national Dutch elections. 

While ostensibly regional in character, the Dutch provincial elections 
nevertheless have a ‘strong national component” (Hietbrink and van Voorst, 
2011: 6) as they determine the composition of the Dutch senate, which is 
indirectly elected by the ‘provincial states’ (provinciale staten). As such they 
can serve as a national case study similar to that of the two other major case 
studies by BuzzFeed News and the NRC Handelsblad that serve as a kind of 
baseline for this one. In addition to provincial elections, only two months 
later, in May 2019, the Netherlands took part in the EU parliamentary elec- 
tions. Given the close proximity of these two elections, and their different 
character, they together provide an opportunity to explore disinformation 
and ‘fake news’ in the media concerning Dutch politics. 

In the following, we first discuss how their methods may be appropriated 
for this case study, through an adapted query list and a more well-defined 
typology of ‘mainstream’ versus ‘junk news’ sources, a term preferred over 
fake news, as we discuss in more detail below. We then analyze the results 
in terms of overall trends and a characterization of the sites found in the 
junk news category. By way of wider contextualization, these findings are 
further compared with results found in other case studies contained within 
this volume. Finally, we offer a characterization of the platform-specific 
and cross-platform trends, and a qualification of the role junk news plays 
in Dutch political news coverage. 


The BuzzFeed method: results so far 


The two aforementioned journalistic analyses that have investigated 
discourse on Facebook in the context of national elections serve as a 


2 Transl. from Dutch: ‘de statenverkiezingen hadden een sterke nationale component’ 
(Hietbrink and van Voorst, 2011: 6). 


74 STIJN PEETERS AND RICHARD ROGERS 


methodological starting point here. These are BuzzFeed News’ landmark 
report into ‘fake news’ in the lead-up to the US presidential elections of 
2016, and the NRC Handelsblad’s study of news shared on Facebook around 
the Dutch parliamentary elections of 2017, which was inspired by BuzzFeed 
News’ report and to a large extent employed the same method. 

Both of these studies used BuzzSumo, a commercial content aggregation 
and analysis platform, to track the most engaged-with articles shared on 
Facebook in the chosen time period. BuzzSumo defines ‘engagement’ as a 
‘sum of likes, comments, and shares attributed to an article’ (Lee, 2019). If the 
article is shared in multiple places (e.g., in multiple groups), the engagement 
score represents the sum of all engagement that BuzzSumo has gathered 
from the platform. After capturing this data through BuzzSumo, both 
BuzzFeed News and the NRC Handelsblad categorized the results as of one of 
two categories, ‘mainstream’ and ‘fake news’. This simple typology has the 
advantage of providing clear results, though is potentially limited through 
its lack of nuance in terms of distinguishing between disinformation, con- 
spiracy, clickbait, and hyperpartisan (as discussed in the introduction to 
this volume), or related terms as problematic information, misinformation 
and mal-information. 

We adopt this basic method for our case study, but some refining is offered 
as the original description could be said to lack specificity in some areas. 
Particularly, with regards to what BuzzFeed News considers ‘fake’, the report 
is somewhat ambiguous, but it does provide the source list in the form of 
open data. On the one hand, BuzzFeed News consistently refers to content 
as either ‘mainstream’ or ‘fake’/‘false’, implying that all of the content in 
that category constitutes articles containing untrue information. On the 
other hand, their definition of ‘fake’ is somewhat expansive in the sense that 
hyperpartisan sites such as Breitbart News are included in their ‘fake news’ 
category. Either way, the most engaged with content they found primarily 
consisted of such false stories as the Pope endorsing Donald Trump, Hillary 
Clinton selling weapons to ISIS, and a fabricated ‘leaked email’. 

While the NRC Handelsblad’s study broadly uses the same approach, 
its method differs in how it categorizes the articles it found. Rather than 
focusing on ‘fake’/‘false’ news, the NRC Handelsblad uses a broader category 
of ‘news that is taken out of context, strongly politically coloured, or has a 
strongly exaggerated headline” (Kist and Zantingh, 2017). Approximately 
10% of the content they found fit this description. This would include 


3 Transl. from Dutch: ‘nieuws [dat] uit zijn context werd gehaald, sterk politiek gekleurd werd 
gebracht of werd voorzien van een sterk aangezette kop’ (Kist & Zantingh 2017). 
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hyperpartisan outlets, even if they do not make false claimsin their content. 
Their report notes that very little of the content they found was actually 
false news, or consciously misleading, but that approximately 10% of the 
content they found fit the broader description. Crucially, even with this 
broader definition their ‘non-mainstream’ category is far smaller than that of 
BuzzFeed News’ findings, and thus the NRC Handelsblad answers its question 
of whether fake news (‘nepnieuws’) plays a role in Dutch elections with a 
resounding ‘no’. In spite of these different outcomes, in different contexts, 
both studies follow the same basic methodology of extracting results from 
a number of relevant queries from BuzzSumo, which we follow here. 

BuzzFeed News’ method, as described in their report, is relatively straight- 
forward: a list of queries is prepared, engagement for articles matching 
these articles is extracted from Facebook (via BuzzSumo), the results are 
aggregated and divided into three-month periods, results are coded as 
either ‘fake’ or ‘mainstream’, and the relative prevalence of both categories is 
plotted over time (Silverman, 2016). More practically, this data was collected 
by BuzzFeed News by querying BuzzSumo for a number of thematically 
appropriate queries. While no full query list is given, the examples include 
names of election candidates ([“Hillary Clinton”] and [“Donald Trump”]) and 
phrases reflecting topics of debate during the campaign, such as [Clinton 
AND emails]. They also included a number of ‘known viral lies’ such as 
[Soros AND voting machine]. It should be noted that the latter inclusion 
is somewhat asymmetrical for it means the search for more sensational 
and divisive subject matters is more precise and targeted than the search 
for mainstream news topics, thereby seeking ‘fake news’. In any case, the 
question of asymmetry is addressed in the case study at hand. 


Query design: descriptions, issues and party leaders 
Dutch provincial elections 


We follow BuzzFeed News and the NRC Handelsblad in their general method 
in terms of query design, querying BuzzSumo in order to find the most 
engaged-with content on Facebook. We compiled a list of queries to search 
BuzzSumo following BuzzFeed News’ approach of mixing names of political 
leaders with issues that were particular to the given election campaign. 
This method also was used by the 2017 NRC study which queried ‘words like 


> a 


“elections”, “parliament” and “polls”, and/or the name of a party, party leader, 


nm u 


and/or widely discussed topics such as “health care”, “pensions”, “immigrants” 
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and “EU” (Kist and Zantingh, 2017).* We used the NRC Handelsblad list as 
a starting point and adjusted it to fit the provincial elections rather than 
the national elections they studied. 

A complication here is the dual local/national focus of the elections. While 
candidate lists differ per province, in televised debates, national rather than 
local party leaders participate, and they can generally be said to dominate 
media coverage (though some local broadcasters organize their own debates 
as well). In terms of media coverage, local leaders are simultaneously more 
numerous (as there are far more local leaders than national leaders) and 
much less significant (as news coverage and debates concentrate on national 
leaders). A national focus additionally was particularly apparent in the 
2019 elections as polls indicated the cabinet risked losing a senate majority 
following the elections (Herderscheé and Meijer, 2019). For this reason, we 
limited our party-based queries to the last names of the political leaders 
of the parties that currently constitute the Dutch parliament? as well as 
the name of the Prime Minister, representing the national government.® 

Additionally, we queried a number of political issues that were topics of 
debate during the election campaign. We looked at the manifestos of the 
larger Dutch parties and chose three themes that were both significant across 
all parties’ manifestos and had been the topic of media coverage during the 
ongoing campaign: [Klimaat] (climate), [Migratie] (migration), and [EU]. 
Finally, we queried two further general keywords, [verkiezingen] (elections) 
and [PS2019], a widely used hashtag and shorthand for the elections at hand. 

The queries were undertaken to capture the election campaign period 
from 18 February 2019 (the start of the first full week of campaigning, marked 
by the launch of various voting aids and launch events hosted by a number 
of parties) to 5 March 2019 (five days after the elections), or five full weeks 
after the start of the campaign for the provincial elections 


EU parliamentary elections 


Using the same general strategy, another set of queries was made to find 
discussion pertaining to the EU parliamentary elections on 23 May 2019. 
As parties ran with national lists of candidates in this case, we queried the 


4 Transl. from the Dutch by the authors: ‘termen als “verkiezingen”, “Tweede Kamer” en 
“peiling”, en/of de naam van een partij, lijsttrekker en /of veelbesproken onderwerpen als “zorg”, 
“AOW”, “asielzoekers” en “EU” (Kist and Zantingh, 2017). 

5 [Asscher], [Baudet], [Buma], [Dijkhoff], [Jetten], [Klaver], [Krol], [Kuzu], [Marijnissen], 
[Segers], [Staaij], [Thieme], and [Wilders]. 

6 [Rutte]. 
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lead candidates for each party in addition to the current political leaders of 
all parties in the Dutch parliament.” Querying these again was necessary 
as national leaders played an active role in the election campaign, such as 
when Mark Rutte, the VVD Prime Minister, and Thierry Baudet, the leader 
of the FvD, engaged in a televised debate on the eve of the elections. 

We further queried general election-related phrases, as well as three 
themes that occurred across multiple parties’ manifestos: climate [klimaat], 
migration [migratie] and [privacy]. As the elections coincided with a govern- 
ment campaign seeking to make voters aware of the dangers of disinforma- 
tion (Ministerie van Binnenlandse Zaken en Koninkrijksrelaties, 2019), we 
also queried [“fake news” OR fakenews OR nepnieuws OR desinformatie 
OR junknieuws]. Finally, for this election we also queried the names of all 
parties for which one could cast a vote P 

We queried these key words using BuzzSumo, limiting ourselves to articles 
in Dutch, excluding Belgian sources. As with the Dutch provincial elections, 
for the EU campaign we queried a similar 5-week period between 19 April 
and 23 May (election day). Finally, we removed irrelevant results such as 
those covering various Belgian election campaigns and those resulting 
from ambiguous keywords such as [Klaver], the name of a party leader but 
also the word for clover. 


Outlet coding: fake and/or junk news? 


An important question here is how one identifies a source as either 
mainstream or its counterpart, whether fake news, junk news or an- 
other term (such as problematic information). While mainstream appears 
rather straightforward to identify (though that also may shift in time), 
its counterpart is a fuzzier concept. BuzzFeed News described their ‘fake 
news’ as emanating ‘from news websites that only publish hoaxes or from 
hyperpartisan websites that present themselves as publishing real news! 
(Silverman, 2016). Here both types of sites purport to be ‘news’, but not in 
the manner or with the substance that the mainstream publishes, given 
their hoaxes or hyperpartisanship, or strong political colour. 


7 [“De Graaff”), [“De Lange"), [“in ‘t Veld”), van Dalen”), [“van der Spek”], [“van der Staaij”], 
(“van Lanschot”), [Asscher], [Azmani], [Baudet], [Berendsen], [Buma], [Dijkhoff], [Eickhout], 
[Eppink], [Hazekamp], [Hoekstra], [Jetten], [Klaver], [Krol], [Kuzu], [Manders], [Marijnissen], 
[Rutte], [Segers], [Thieme], [Timmermans], [Tonga], [Wierda], [Wilders]. 

8 [soPlus], [CDA], [Christenunie OR SGP], [D66], [Denk], [FvD OR “Forum voor Democratie”], 
[GroenLinks], [“Jezus Leeft", [PvdA], [PvdD OR “Partij voor de Dieren"), [PVV], [SP], [VVD]. 
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Another notion is ‘junk news’, and it may be preferred because it avoids 
the other, historically fraught ‘fake news’ definition of the ‘lying media’, but 
is more ontologically flexible, at least as scholars have described it. While 
this term has been used as a synonym for ‘fake news’ (Venturini, 2019: 10), 
Marchal et al. (2018) employ it to capture a broader category of content 
that consists of ‘various forms of propaganda and ideologically extreme, 
hyperpartisan or conspiratorial news and information’ (2). This then would 
include BuzzFeed News’ notion, but also part of the NRC Handelsblad’s 
broader category of tendentious sites that may more often comment upon 
rather than deliver news, as we come to. 

For their Junk News Aggregator’, a Facebook junk news scraping project, 
researchers at the Oxford Internet Institute identified a set of measures to 
define what qualifies as junk news, consisting of 1) a lack of journalistic 
standards; 2) tendentious style; 3) low credibility; 4) clear bias; 5) a mimicry 
of traditional news reporting aesthetics; or 6) aggregating content matching 
the first five criteria (Liotsou et al., 2019: 3). A source was then considered 
junk news if it satisfied at least three of the first five criteria, or the sixth. 
Herein lies the flexibility, but also the breadth of the definition that may 
be suitable for the current analytical purposes in the Dutch case. 

In its report, the NRC Handelsblad concluded that propaganda or disinfor- 
mation did not play a significant role in Dutch media. It also distinguished 
between mainstream and hyperpartisan sources, where the latter is news 
that is purposively taken out of context, exaggerated to promote a cause 
De, tendentious) or strongly politically coloured. A number of Dutch outlets 
can be qualified as both ‘tendentious’ and strongly politically coloured, 
while also being embedded in the Dutch media landscape (and in that 
sense mainstream or mainstreaming). Originally a so-called ‘shock blog’, 
Geenstijl describes itself as tendentious, and gave birth to PowNed, a public 
TV broadcaster with a similar signature style. Given its durability and 
link with the public broadcasting company, GeenStijl could be considered 
both tendentious and mainstream, or the hybrid category, tendentious- 
mainstream. Another case that is prominent in the BuzzSumo results we 
found is The Post Online (TPO). It is a right-wing media outlet and could fit the 
NRC Handelsblad’s definition as well as a broader definition of hyperpartisan 
sites as ‘openly ideological web operations’ (Hermann, 2016). Putting it in the 
same category as more fringe sites such as Ninefornews (a site promoting 
conspiracies and UFOlogy) or De Dagelijkse Standaard (a far-right outlet 
that regularly publishes anti-immigrant articles) would not do justice to 
the less extreme tone. Thus, we could dub it tendentious-hyperpartisan. 
In the analyses to follow here we show the results with tendentious as a 
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separate category made up of these two sources. In other studies to follow 
(on Twitter), the results are compared when the tendentious-hyperpartisan 
source is categorized as either tendentious or hyperpartisan (see Niederer 
and Groen, this volume). 

In the following we employ the fine-grained categorization and con- 
tinuum, distinguishing between ‘mainstream’, ‘tendentious’, ‘hyperpartisan’, 
‘conspiracy’ and ‘clickbait’, occasionally linking the categories, as mentioned. 
These categories reflect the various sub-types of mainstream, tendentious 
and otherwise lower-quality content discussed in the introductory chapter. 
This also allows more nuanced categorizations of sites such as GeenStijl 
and The Post Online. In the following analysis we offer this five-category 
coding as an addition to the binary OlJ-based categorization, as a way to 
illustrate the make-up of non-mainstream content found in the data. This 
categorization resulted from a collaborative coding effort across all case 
studies found in this volume and provides a more detailed alternative to 
the binary ‘fake/junk’ versus ‘mainstream’ opposition found in, for example, 
the BuzzFeed News and NRC Handelsblad studies. 

In all we therefore elect not to reduce the sources to fake but rather use 
a more inclusive category of ‘junk news’, but then also pay special attention 
to the tendentious outlets. After identifying the sites using this typology, we 
further removed all other sites from the results that were either marginal 
or local. Marginal here refers to sites that received very low engagement 
scores in the BuzzSumo results and were not otherwise notable in terms of 
content or overall engagement. We also excluded local news sites, as our main 
concern for this analysis are outlets with a national or otherwise substantial 
reach; regional outlets conversely typically have a limited audience, and 
our list of ‘junk’ sites contained more nationally oriented outlets rather 
than regional ones. This left a ‘mainstream’ category containing national 
outlets, mostly firmly embedded in the Dutch media landscape, such as 
various national newspapers, TV broadcasters and a number of online news 
sites and magazines. 


Data analysis: overall and per-query trends 
Dutch provincial elections 
We used the annotated source list (or expert list) to code the results for 


the BuzzSumo queries, as discussed in more detail below. This allows for a 
per-query observation of the ratio between mainstream and junk sources. 
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Figure 2.1 Engagement of mainstream (blue) and junk-like news (pink) articles 
found through provincial elections-related BuzzSumo queries, per 
week, between 18 February 2019 and 25 March 2019. Engagement 
scores have been normalized. 
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Next to these separate analyses we also calculated an average ratio, weighted 
by the relative engagement per category, on both an overall and a per-week 
basis. While our categorization method is slightly different from BuzzFeed 
News’, this per-week analysis nevertheless allows for a trend comparison with 
the results of their over-time analysis of the US 2016 presidential election 
campaign. 

Notably, the trendline found in our over-time analysis (Figure 2.1) does 
not match the one in BuzzFeed’s study (see Figure 2.2). While BuzzFeed 
News’ data saw a clear increase of engagement of fake news in the weeks 
leading up to the elections, in our data junk news stayed relatively constant 
in terms of engagement and even decreased slightly during the last few 
weeks. There are, however, some differences between the two campaigns 
that complicate a direct comparison. The US election campaign is typically 
far longer than Dutch election campaigns, especially in this case as the 2019 
election was concerned with the provincial states and senate rather than the 
lower house of parliament (typically the most important Dutch election). 
While the US campaign was analyzed over a period of 9 months, the Dutch 
campaign and hence our data spans five weeks only. Additionally, BuzzFeed 
News’ data resolution is quite low (one datapoint per three months) while 
ours is more fine-grained (one per week). 

Nevertheless, even considering these differences it is striking that the 
graphs indicate rather different dynamics. While the BuzzFeed data points 
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Figure2.2 Total Facebook engagement of fake versus mainstream news. Results 
from election-related queries on BuzzSumo, for the 20 most-engaged 
with articles between February and November 2o16, per three-month 
period 
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Source: Silverman (2016) 


to a clear ramping up of fake news content as the election date draws near, 
our data is more in line with the NRC Handelblad’s earlier study and suggests 
a more constant but persistent undercurrent of junk news that is a part 
of politically oriented media output. The above data is an aggregate of all 
queries performed on BuzzSumo, however. While in aggregate there is no 
clear trend, this could be the result of summing up the values, and more 
apparent trends exist in the results for individual queries. 

As can be seen in the overview in Figure 2.3, even on a per-query basis 
there are few clear trends with regards to the prevalence of junk news 
engagement. There is an interesting uptick in the prevalence of mainstream 
engagement for a few queries. Most notably, the data for [Segers], the leader 
of ChristenUnie (a centrist Christian party), shows a sharp increase in the 
last week of the election campaign. This can almost entirely be attributed, 
however, to news coverage after the elections about the implications of the 
election results for the cabinet, of which Segers’ party is the smallest member. 
(Note the similar uptick for [Jetten], whose D66 party is the second-smallest 
cabinet member.) Another notable bump in mainstream engagement occurs 
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Figure 2.3 Per-query engagement of mainstream (blue) and junk (pink) articles 
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found through provincial elections-related BuzzSumo queries, per 


week, between 18 February and 25 March 2019. Engagement scores 


have been normalized. 
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for a number of queries ([PS2019], [Buma], [Kuzu] and [Dijkhoff |) around the 
middle of the election campaign. A closer look at the articles responsible 
for this engagement reveals that this may be an indication of the campaign 
coming into full swing and hence the increasing media coverage of it. The 
oft-quoted and feared BuzzFeed News pattern of fake news outperforming 
mainstream news is thus not repeated on either an aggregate or query level 


in this case study. 
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What remains ofinterest is the relative performance of mainstream and 
junk news on a per-week and per-query level, particularly on a number of 
occasions where junk news briefly outperforms mainstream news in terms 
of Facebook engagement. For queries of politicians, it occurs most notably 
for [Baudet], [Kuzu] and [Wilders] during the first week of the election 
campaign, where the dominance of junk news is most pronounced. These 
politicians all lead relatively fringe parties: Baudet leads the far-right Forum 
voor Democratie (FvD), Kuzu the left-wing and immigrant-oriented DENK, 
and Wilders is the leader of the far-right Partij voor de Vrijheid (PVV). A 
closer look at the junk news articles that are responsible for these surges 
shows that in all three cases, these are not articles primarily concerned 
with the elections themselves but rather coverage of other political issues 
(mostly around climate laws that were being discussed at the time) by 
hyperpartisan outlets like De Dagelijkse Standaard. The relative prevalence 
of this coverage is perhaps an indication that media had not yet started 
covering the election campaign in earnest, rather than a dominance of 
junk news in election discourse. Overall, while in some individual cases 
junk news outperforms mainstream news, these episodes are outliers and 
represent less of an overall trend than one for particular parties. There is 
one general exception to this rule, however, and it concerns the query for 
[migratie], or migration, where junk outperforms mainstream for most of 
the period. Also, [klimaat] or climate, has a week where junk news had 
more engagement that the mainstream. These are rather polarizing issues, 
drawing attention from hyperpartisan outlets. 


EU Parliamentary elections 


A trend analysis of the EU parliamentary results (see Figure 2.4) shows a 
pattern not too dissimilar to the one found in the provincial elections data, 
similarly seeing junk news match the performance of mainstream news 
particularly in the beginning of the query period. Recall that during the 
provincial elections campaign junk news performed as well as mainstream 
news on two occasions. Though this trend is still notably different from the 
one found by BuzzFeed News, where junk news overtook mainstream news 
towards the end of the campaign, it is nevertheless a significant finding that 
suggests an increasingly robust position for junk news in the Dutch context. 

A closer look at this second week of the EU campaign data shows that 
the junk news engagement can for a large part be attributed to an article 
in De Dagelijkse Standaard, which discusses a video posted by the political 
party Denk on their Facebook page, accusing the party of demonizing 
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Figure 2.4 Engagement of mainstream and junk-like articles found through EU 
elections-related queries on BuzzSumo, between 19 April 2019 and 


23 May 2019. Engagement scores have been normalized. 
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Geert Wilders (of the PVV party).9 This article's engagement is responsible 
for about 36% of that week’s ‘junk’ engagement, providing a major boost. 

More generally the relatively high engagement attained by junk sources 
can in many cases be attributed to a small number of high-performing 
articles. This matches the findings from the analysis of the provincial elec- 
tions, where peaks in junk news engagement could similarly be attributed 
to a smaller number of well-scoring articles. While junk sources perform 
relatively well, especially in the earlier weeks of the data set, this success is 
thus attributable to a relatively small number of sources and articles rather 
than a broadly successful and diversified ecosystem or even a coordinated 
campaign. 

Though the findings do not approximate those in the BuzzFeed News story, 
in the case of the EU election campaign it is noteworthy that indeed junk 
news does on one occasion match the performance of mainstream news, 
though not during the tail end of the campaign period as was the case in 
the BuzzFeed News data. Overall, junk news is roughly as successful during 
the EU campaign as it was during the provincial election campaign, and 
has a significant presence, though over the whole campaign mainstream 
news still easily outperforms it. 


9 ‘Video! Kuzu (DENK) wil dat Wilders gestopt wordt, voordat hij een tweede Srebrenica- 
bloedbad kan aanrichten’: https://www.dagelijksestandaard.nl/2019/05/video-kuzu-denk-wil- 
dat-wilders-gestopt-wordt-voordat-hij-een-tweede-srebrenica-bloedbad-kan-aanrichten/ 
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found through EU parliamentary election-related BuzzSumo queries, 
per week, between 19 April 2019 and 23 May 2019. Engagement scores 


Figure 2.5 Per-query engagement of mainstream (blue) and junk (pink) articles 
have been normalized. 


POLITICAL NEWS ON FACEBOOK DURING THE 2019 DUTCH ELECTIONS 


SS = 
° S o 
23/05/19 + 230509 2 g 23/05/19 + 23/05/19 o 23/05/19 . 23/05/19 @ 230519 9 23/05/19 
17/05 + 17/05 ° š 17/05 e + 17/05 2 17/05 - 17/05 ° 17/05 N 17/05 
s 10/05 x 10/05 exe , 10/05 ° 10/05 G 10/05 10/05 k 10/05 3 10/05 
£ 
E oam S mus Š Š oso & 03/05 5 mom Q osos $ osos $ 03/05 
E 26/04 a 26/04 SS 26/04 ki 26/04 ke 26/04 > 26/04 > 26/04 > 26/04 
Ss Ee e ° o ə ° o g g o e ° ° o ° o e ° o ə ° o 
8 8 8 8 & 8 a S & S 8 8 & 8 R 8 
° 
3 
E K e 
E ° e D 
© 23/05/19 @ 23/05/19 4 © 23/05/19 ç 23/05/19 23/05/19 «es 23/05/19 23/05/19 e 23/05/19 E 
3 1705 8 1705 x 1705 5 17/05 | 705 688 17/05 1705 © 17/05 SS 
5 10/05 Š 10/05 S ° 1005 “Š 10/05 3 10/05 a EP 10/05 10/05 > - 10/05 £ ° 
= 
s 03/05 5 03/05 3 8 03/05 SZ 03/05 DE S, 03/05 a 03/05 Z 03/05 EX 5 
a 26/04 Q 26/04 WN 26/04 I 26/04 26/04 å a 0 26/04 o 26/04 £ 26/04 £ 5 D 
2 g ° ego e g ° e ° o e gege e e o ə o o e ° o | | 
S S & S 8 8 8 8 8 288 8 38 & S 8 S 
© 
a °, 
° a DE 
23/05/19 5 23/05/19 5 23/05/19 E 23/05/19 23/05/19 23/05/19 23/05/19 5 © 23/05/19 
H 
1705 S 17/05 Uü 17/05 G 17/05 1705 > 17/05 e 17/05 cs 17/05 
10/05 A 10/05 8 10/05 5 10/05 _ 10/05 8 10/05 © 10/05 tas 10/05 
03/05 Š 03/05 Sa 03/05 03/05 ° 03/05 z 03/05 p 03/05 5 = e 03/05 
2604S 26504 WO 265/04 I 2604 x 26/04 G 26/04 0 26/04 >00 26/04 
a a EI a ° ° a a ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° 
& 8 S 8 a S 8 S 8 8 & 8 8 8 a 8 
a £ 
23/05/19 - 23/05/19 23/05/19 £ 23/05/19 - 23/05/19 8 23/05/19 23/05/19 23/05/19 23/05/19 
© 17/05 . 17/05 x 17/05 3 17/05 Ñ - 17/05 2 17/05 17/05 17/05 E 17/05 
£ 10/05 . 10/05 E 10/05 £ 10/05 © . wos E 10/05 © 1005 8 10/05 © 10/05 
° © a Š S 2 = £ Ë = 
2 ms S oss ` S 03/05 0305 mom se 03/05 5 0305 § 03/05 = 03/05 
< 25/04 Q 26/04 u 26/04 © 265/04 & 26/04 = 26/04 Œ 26/04 F 26/04 z 26/04 
ə ° o ə o o e ge e geee e g ° ə ° o ə ° o es sg o 2o ° o 
8 3 R 8 8 S ə 888 S 8 a 8 & 8 8 8 S 8 
a = £ Be = Ë ë = = 
a 
£ 5 
"age 5 23/05/19 — 23/05/19 Ë 23/05/19 + 23/05/19 „ 23/05/19 „ 23/05/19 £ . 23/05/19 23/05/19 
z 17/05 5a 17/05 2 vos $ 17105 4 . 105 5 17/05 - 17/05 5 . 17/05 g 17/05 
© 10/05 a g 10/05 £ 10/05 E 10/05 10/05 g 10/05 10/05 £ - 10/05 s 10/05 
E 03/05 Ec 03/05 O 03/05 o 03/05 = h 03/05 e 03/05 z 03/05 £ + 03/05 Zë 03/05 
2 2604 6G zeis Qi ed "8 26/04 x 26/04 Š 26/04 & 26/04 E 26/04 z 26/04 
e g ° ° e e o e o 2 ey © e g ° e o e e ë e e g e e e o 
& S g 8 8 8 Soe 8 8 8 8 g 8 8 8 g 8 
z 
Š 
» 
52 £ 2 
23/05/19 23/05/19 23/05/19 © 5 23/05/19 3 23/05/19 Q 23/05/19 + 23/05/19 23/05/19 23/05/19 
= 
° 17/05 17o $ 17/05 Z5 17/05 4 17/05 S 17/05 17/05 © 17/05 | 17/05 
z 10/05 gg 10/05 £ 1008 S £ 10/05 s 10/05 A 10/05 < 10/05 £ 10/05 x 10/05 
a 03/05 A 03/05 = 0305 Ss 03/05 H DOE S oso g oam “= 03/05 5 03/05 
Wm 26/04 O 26/04 Q 26/04 LA 26/04 7 26/04 > 2604 & 26504 F 26/04 > 26/04 
° ° o ° ° o e e o e ° o e ° o e ° o e ego g se o e ° o 
& & 8 8 8 8 & 8 8 8 S 8 & 8 8 8 & 8 


visualization by Federica Bardelli 


u 


Line graphs 


86 STIJN PEETERS AND RICHARD ROGERS 
Characterizing sources 


It is useful here to briefly discuss the sites that make up both categories of 
content. Our category of mainstream outlets (see Table 2.1) consists of well- 
known outlets with a national reach, which in practice translates to a number 
of national newspapers, public broadcasting organizations, national TV 
programmes and large online magazines. The junk category is comparatively 
more diverse; the typology we use covers conspiracy sites, hyperpartisan 
online sources (including independent self-styled journalists), and clickbait 
aggregators. Some of these are relatively large: De Dagelijkse Standaard, a far- 
right weblog, appears in the top three of most engaged-with articles for 15 of 
our 1g queries. Some other junk sites appear to be more focused on a particular 
topic; this is especially apparent in the results for the provincial elections 
[Migratie] (migration) query, in which fenixx.org — a far-right extreme site 
advancing the ‘race replacement’ theory — appears often, while it is far less 
prominent for the other queries, save for the [EU], in which it also appears 
occasionally. This site was also noted by the earlier 2017 NRC Handelsblad 
study as being especially prevalent in their ‘hyperpartisan’ category. 

This ‘hyperpartisan’ category can then be seen to be comprised of roughly 
the same set of sites in both data sets (see Table 2.1 and 2.2). This could be 
considered to suggest a hyperpartisan news ecosystem of sites that enjoy a 
significant and stable readership. On the other hand, this ecosystem is notably 
top-heavy; for both data sets De Dagelijkse Standaard (DDS) is by far the most 
engaged-with site, almost four times as popular as the next site in the list. 
Following DDS is a number of far smaller but simultaneously more outspokenly 
far-right blogs such as Stop de Bankiers, Fenixx and JD Report. Fenixx here is 
further notable as a site that was also mentioned as a relatively prominent 
junk site in the 2017 NRC study. While we can thus identify a stable sphere 
of hyperpartisan news sites that drive significant engagement, the success 
of this sphere is still mostly reliant on De Dagelijkse Standaard, and with the 
exception of that site is quite marginal compared to the mainstream sphere. 

As discussed above, an alternative to the binary mainstream/junk op- 
position one may consider the data for both election campaigns in terms of a 
more detailed five-category perspective (see Figures 2.5 and 2.6). For both the 
provincial and EU elections it is apparent that the largest non-mainstream 
category by far consists of hyperpartisan sources. The only other category 
that has a noteworthy impact are tendentious sources GeenStijl and The 
Post Online (which are both not included in the other, binary, categorization 
in Figures 2.1 and 2.4). Conspiracy and clickbait sources are present in the 
data but do not play a significant role compared to the other categories. 
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Table 2.1 Top 10 sites per category (provincial elections), for all queries combined, 
sorted by overall engagement scores as reported by BuzzSumo 
Mainstream Junk news 

Site Engagement Site Engagement 
telegraaf.nl 102117 dagelijksestandaard.nl 98414 
nu.nl 46962 stopdebankiers.com 26429 
rtInieuws.nl 46849 fenixx.org 13024 
wnl.tv 39975 jdreport.com 8564 
nos.nl 37319 ninefornews.nl 5975 
nrc.nl 16010 tpook.nl 4431 
metronieuws.nl 14746 ejbron.wordpress.com 4126 
pauw.bnnvara.nl 10130 opiniez.com 2777 
evajinek.kro-ncrv.nl 7412 dimplus.nl 2110 


Table 2.2 Top 10 sites per category (EU parliamentary elections), for all queries 
combined, sorted by overall engagement scores as reported by BuzzSumo 


Mainstream Junk news 

Site Engagement Site Engagement 
telegraaf.nl 232327 dagelijksestandaard.nl 225006 
nu.nl 192962 stopdebankiers.com 46892 
nos.nl 141440 fenixx.org 25852 
rtInieuws.nl 99820 tpook.nl 17453 
wnl.tv 91211 jdreport.com 9199 
elsevierweekblad.nl 31150 opiniez.com 8302 
metronieuws.nl 28038 ejbron.wordpress.com 6427 
nrc.nl 27195 reactnieuws.net 5565 
joop.bnnvara.nl 22509 ninefornews.nl 2047 


Table 2.3 Top 10‘hyperpartisan’ sites for both data sets (provincial and EU 
elections), sorted by overall engagement scores as reported by BuzzSumo 


Dutch provincial elections 


EU Parliamentary elections 


Site Engagement Site Engagement 
dagelijksestandaard.nl 168668 dagelijksestandaard.nl 225006 
stopdebankiers.com 35414 stopdebankiers.com 46892 
fenixx.org 20757 fenixx.org 25852 
jdreport.com 15679 jdreport.com 9199 
ejbron.wordpress.com 5285 opiniez.com 8302 
dailypaper.org 4887 ejbron.wordpress.com 6427 
opiniez.com 4554 reactnieuws.net 5565 
destaatvanhet-klimaat.nl 3912 xandernieuws.net 2009 
pallieterke.net 3228 eunmask.wordpress.com 1296 
eunmask.wordpress.com 2487 novini.nl 862 
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Figure 2.6 Engagement of mainstream, hyperpartisan, conspiracy and clickbait 


articles found for provincial elections-related queries on BuzzSumo, 


between 18 February 2019 and 25 March 2019. Engagement scores have 


been normalized. GeenStijl is considered ‘mainstream’ here, while The 


Post Online is classified as ‘hyperpartisan’. 
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Figure 2.7 Engagement of mainstream, tendentious, hyperpartisan, conspiracy 


and clickbait articles found for provincial elections-related queries on 


BuzzSumo, between 18 February 2019 and 25 March 2019. Engagement 


scores have been normalized. GeenStijl and The Post Online are 


considered ‘tendentious’ here. 
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Figure2.8 Engagement of mainstream, tendentious, hyperpartisan, conspiracy 
and clickbait articles found for EU parliamentary elections-related 
queries on BuzzSumo, between 19 April 2019 and 23 May 2019. 
Engagement scores have been normalized. GeenStijl is considered 
‘mainstream’ here while The Post Online is classified as ‘hyperpartisan’. 
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Figure2.9 Engagement of mainstream, tendentious, hyperpartisan, conspiracy 
and clickbait articles found for EU parliamentary elections-related 
queries on BuzzSumo, between 19 April 2019 and 23 May 2019. 
Engagement scores have been normalized. GeenStijl and The Post 
Online are considered ‘tendentious’ here. 
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An examination of the most engaged-with sites in the hyperpartisan 
category (see Table 2.3) further confirms that this category is the most 
influential one in the broader ‘junk’ (or ‘junk-like’) sphere, with the top ten 
sites mostly matching those found in the top 10 of ‘junk’ sites identified in 
tables 1 and 2. The top five is similar on all lists, and again De Dagelijkse 
Standaard is the most important site. Notably, as the campaign draws on, 
mainstream engagement can be seen to increase while junk news perfor- 
mance is relatively stable, meaning interest in mainstream news coverage 
increases towards the end of a political campaign, while junk news remains 
stable. Perhaps they serve different publics, though such a construal would 
require further work. 

Generally, the junk news sites, of which hyperpartisan sites are the largest 
constituent, can be characterized as on the right, anti-immigrant, anti-EU 
and in some cases anti-Semitic or advancing conspiracy theories (the latter 
especially applying to ninefornews.nl and jdreport.com). This ideological 
slant in our findings is consistent with other studies on junk news, including 
the 2016 BuzzFeed News analysis but also others that found that left-wing 
content was less prominent in that category (Silverman, 2016; Neudert et 
al., 2017: 1; Alcott and Gentzkow, 2017: 223). In this case study, next to the 
prevalence of hyperpartisan sites such as DDS the relatively large engagement 
of especially conspiracy sites is notable; ninefornews.nl, which is the sth- 
most engaged with site in our data, regularly promotes conspiracy theories 
ranging from UFO sightings to such far-fetched concoctions as Pizzagate and 
QAnon. The authors seem to be convinced that this is accurate accounting 
of events. Overall, the data show that junk news, consisting primarily of 
hyperpartisan and conspiracy theory sites, are a minor but constant and 
significant factor. 


A cross-platform appraisal 


This case study focuses on Facebook, but a similar analysis may be performed 
for other platforms. While Facebook has the dubious honour of being the 
platform with perhaps the strongest association with fake news, other 
platforms have their own affordances that could make them attractive for 
those seeking to spread forms of junk content. Just as this case study builds on 
the analyses of BuzzFeed News’ and the NRC Handelsblad's, with a number of 
methodological tweaks, one could similarly move to other platforms as well, 
studying over-time engagement of junk and mainstream content respectively. 
Multiple case studies in this volume employ a method of this type. 
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The multiple platform analyses present an opportunity not only to 
investigate the prevalence of junk news on individual platforms, but also 
to perform a cross-platform analysis in order to investigate whether there 
are platforms that are particularly susceptible to junk content, or whether 
some platforms have perhaps succeeded in combating the spread of it, given 
that the phenomenon has been addressed for some time now, and the case 
studies in question take place in early to mid-2019. While we present such 
a comparison in this section, it should be noted that a direct comparison 
between platforms is complicated for a number of reasons. 

One issue with a comparison between various platforms is that ‘engage- 
ment’ means different things depending on the features a platform offers 
for interacting with content. On Facebook, engagement means the sum of 
comments, likes (or reactions) and shares a post received. But Reddit, for 
example, has no direct counterpart to some of those, as ‘shares’ are not a 
relevant concept on that platform. It simultaneously offers metrics Facebook 
does not use (including upvotes and downvotes). Moreover, on Facebook 
a dislike or angry reaction, for example, often would be counted as a plus 
engagement, whereas a downvote on Reddit reduces a post’s score. 

More specifically, the case studies in this collection use different time 
periods and, in some cases, investigate, apart from election issues and 
leaders, certain polarized topics (such as MH17 and Zwarte Piet) so as to 
seek disinformation or junk, as we discussed above in terms of asymmetrical 
querying. Such query design may be justified, given that previous studies 
of disinformation in the Dutch media context were borne of data curated 
by Twitter that consisted of Russian IRA trolls, and found activity around 
the downing of the MH17 airliner in 2014 as well as the terrorist attacks 
in the Brussels airport and metro in 2016. When examining on Twitter 
the MH17 hashtag and keyword usage over the past number of years, one 
may find increased activity around elections (such as during the national 
elections of 2017), thus further justifying a renewed attention towards at 
least MH17 during the 2019 elections. Such asymmetrical querying of course 
complicates comparisons, as the ratio between mainstream and junk news 
engagement may be less balanced, given conspiracy and other sources’ 
continual attention to such themes. Differences in time periods also pose 
issues, as there may be particularly ‘junk-sensitive’ episodes from the past 
that are missing from the current analyses, and for analytical purposes 
have been removed from the comparison. 

Nevertheless, provided one is aware of the limitations in such a com- 
parison, the results of such an analysis for other platforms compared to the 
Facebook case study can provide an impression of the relative penetration of 
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Figure 2.10 Relative engagement of content categories across 4chan /pol/, Reddit, 
Twitter and Facebook. GeenStijl is considered ‘mainstream’ here while 
The Post Online is classified as ‘hyperpartisan’. achan and reddit data 
from 1 Dec 2015 until 1 June; Twitter and Facebook data from 18 Feb 
2019-25 Mar 2019 and 19 Apr 2019-23 May 2019 
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Figure 2.11 Relative engagement of content categories across 4chan /pol/, Reddit, 
Twitter and Facebook. achan and reddit data from 1 Dec 2015 until 
1 June; Twitter and Facebook data from 18 Feb 2019-25 Mar 2019 and 19 


Apr 2019-23 May 2019 
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junk across different social media platforms. While in the rest of this volume 
there are separate case studies that investigate the individual platforms with 
methods similar to this one, the graphs above present a rough impression 
of the results across platforms, using data from this chapter and the other 
case studies. 

What is striking in the cross-platform comparison of results in Figures 2.10 
and 2.11 is that the two ‘mainstream’ social platforms, Facebook and Twitter, 
show a higher prevalence of junk content than 4chan and Reddit, the deep 
vernacular web platforms. This is interesting because the latter two — the 
‘seedy underbelly’ of the internet (Bergstrom, 2011) — are often characterized 
as hotbeds of polarizing and alt-right political discussion, thus providing an 
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environment where one could expect particularly hyperpartisan content 
to thrive. 

One plausible explanation of this is that especially on 4chan’s /pol/, the 
‘politically incorrect’ sub-forum that was investigated in this case, those 
posting may position themselves in opposition to mainstream sources. 
This positioning often goes hand in hand with linking to the sources in 
question, thus increasing the share of mainstream content in the overall 
picture for the platform. As such it underlines the notion that engagement 
does not necessarily indicate that one agrees with the engaged-with content, 
and in fact high engagement may be taken to indicate controversiality, as 
something polarizing that is hotly debated can be expected to be clicked 
on and scrutinized by many of those posting about it. 

As discussed, a detailed cross-platform comparison is complicated by 
the different methods used in each case study. While outside the scope of 
this research, further commensuration of these methods and results for a 
more thorough cross-platform analysis presents an opportunity for further 
research. 


Conclusions: Absence of disinformation and junk news prevalence 


This particular case study, focused on Facebook, is informed by similar 
investigative (data) journalism originating with BuzzFeed News and the 
NRC Handelsblad. Our results are not strictly in keeping with theirs. It is of 
particular interest that the prevalence of intentionally false news BuzzFeed 
News found was not apparent in our data, indicating that this is far less of a 
problem in the Dutch sphere than in the US. The data do seem to confirm 
the reputation of Facebook as an especially fertile ground for junk news in 
comparison to other platforms and indicates that despite its initiatives to 
combat such content, it is still endemic on the platform. In fact, whereas the 
NRC found that ‘at most 10%™ (Kist and Zantingh, 2017) of the engagement 
they analyzed concerned hyperpartisan and tendentious content, in our 
analysis a little over a year later we find this share has risen to 25%. 
While this difference between our findings and the NRC Handelsblad’s 
could partially be explained by the differences in the criteria used to catego- 
ries the content, it seems justified to conclude that even ifjunk news is in the 
minority, it is certainly not marginal, and seems to be a growing product in 
the Dutch media landscape, on some occasions matching the performance 


10 Transl. from Dutch by authors of ‘hoogstens 10 procent’ (Kist & Zantingh, 2017). 
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of mainstream news in terms of Facebook engagement. Though this case 
study is limited to the 2019 provincial and EU elections, its findings suggest 
that a broader analysis of junk coverage of Dutch politics on Facebook is 
warranted. Such an analysis could also investigate what ‘engagement’ means 
in practice; as indicated by the cursory cross-platform analysis, engagement 
may not translate to agreement, and if junk news is such a factor on Facebook 
it is important to understand the motivations behind engaging with it if 
we are to understand the significance of it in the wider political debate. 

A silver lining (so to speak) is that there was virtually no outright (foreign) 
disinformation in the data we found, and indeed across all platforms we 
investigated. While especially on Facebook there is a solid undercurrent of 
junk sites including hyperpartisan content, and a number of well-shared 
conspiracy sites which promote highly dubious content, there is no imminent 
reason to expect so-called fake news affecting Dutch election coverage in 
the same way it appeared to for the 2016 US elections. Overall, our Facebook 
case study indicates that there is no immediate cause for concern about 
disinformation about Dutch elections, but that junk news is a growing 
factor that warrants closer scrutiny. 
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3 Politicalnewsin search engines 


Exploring Google's susceptibility to hyperpartisan sources 
during the Dutch elections 


Guillén Torres and Richard Rogers' 


Abstract 

The research enquires into the susceptibility of Google's search engine 
to provide users with questionable information when querying political 
parties and their issues during the run-up to the Dutch provincial and 
European parliamentary elections. Which rankings has the search engine 
assigned to problematic sources when querying political parties and 
theirissues? Are there particular politicalissues and party spaces where 
these sources are prevalent or entirely absent? Do the ranks and amounts 
increase as the elections draw near? Inall,it was found that hyperpartisan 
sources are rather pervasive in the search-demarcated political space, 
but far more so for certain actors and their issues on the far right of the 
political spectrum. 


Keywords: Google Web Search, search engines, elections, social issues, 
digital methods 


Introduction: Search engines as junk source space 


As key entry points to the web, search engines serve as a site for the con- 
sumption of information, including political information, and as such are a 
relevant space for the study of both the presence of disinformation and junk 
news as well as approaches to combat it (Bowden, 2016). Although they are 
described in the industry as ‘organic’, the output of search engines could be 
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termed manufactured hierarchy (Hindman, 2oo8; Halavais, 2o17). Sources 
are ranked per query, and as such certain ones are offered as more relevant 
than others, as if naturally. Such ranking practices are often considered 
inscrutable, since search engines generally do not provide a means to save 
and study query results, e.g., through offering an API that enables it or terms 
of service that allow it. In fact, the ranking algorithms are trade secrets 
jealously guarded by corporations. 

Since 2009 Google and subsequently other search engines have personal- 
ized results, be it for the individual or the place where the search has taken 
place (Pariser, 2011; Puschmann, 2018). Increasingly engines are thus both 
providing ranked political information but also tailoring it to user prefer- 
ences and/or location (Martens et al., 2018). 

When it comes to events, such as elections, search engines become provid- 
ers not only of topical but also of timely information. These are particularly 
poignant moments to study the presence of disinformation and junk news. As 
cases in point, there have been occasions when sources that were otherwise 
insignificantly ranked rose to the top of engine returns during the ‘breaking 
news’ period of an event, such as in the immediate aftermath of the Las Vegas 
shootings in 2017, when a 4chan post misidentifying the shooter rose to the 
top of the results (Robertson, 2017). Google results also prominently linked to 
rumours about the identity of a Texas shooter in 2018 as ‘a Muslim convert, 
member of Antifa or Democrat supporter’ (Lomas, 2018). One could point to 
artificial manipulation, such as search engine optimization, as triggering 
the unexpected rankings and sudden presence of dubious information. 
With respect to the 4chan post, the gaming of the engine (if that were the 
cause) also may have been maliciously playful, introducing misinformation 
as an act of trolling. In the case of the rumours about the Texas shooter, the 
manipulation appears to have been hyperpartisan. Both spates of false news 
were not ‘corrected’ in the editorial sense of an erratum notice; rather, the 
dynamically published results are continually algorithmically tweaked so 
‘good information’ is said to ultimately prevail (Waters, 2017). 

In the study of engine returns and hierarchies (through manual capture 
techniques) it is often pointed out that top placement matters, since engine 
users over the years have been browsing fewer and fewer result pages (Jansen 
and Spink, 2003; Dan and Davison, 2016). Thus, in the above examples, the 
significance of the location of misinformation, rumour and extreme results 
relies on findings about how users gravitate to the top results, making them 
the most consumed and thereby particularly worthy of study. An additional 
research strategy for inquiries into junk news presence concerns anticipatory 
search, also known as autosuggestion, which drives the user to particular search 
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terms. It also has been studied for the offensive associations made by engines, 
such as the completion of ‘are Jews’ with ‘evil’ (Cadwalladr, 2016). Misogynistic 
autosuggestions also were documented in earlier cases which led to a UN 
campaign in 2013 discussed in a longer study of ‘how search engines reinforce 
racism’ (Noble, 2018). Other extremist content has been similarly documented 
towards the top of Google results for the query ‘holocaust’ (Hern, 2017). 

The discovery by The Guardian journalist of the offensive associations 
with the word ‘Jews’ and the resulting sites that surfaced (such as the neo- 
Nazi website, The Daily Stormer) have led to discussions of not just how 
Google’s algorithms could be tweaked, but also the reach of the sources 
producing and driving such information in the first place. Their presence 
(and top placement) could be interpreted as a proxy for the significance 
and audience of such material online, or as others have argued as evidence 
of a culture war, driven by the online boosting tactics of ‘culture hackers’ 
(Albright, 2016; Confessore and Wakabayashi, 2017). Whilst they may seek to 
correct the autosuggestions (and perhaps remove religious ones all together), 
companies such as Google are hesitant to delist such extreme websites, given 
free speech concerns, which also may arise if they are nudged downwards. 

The present chapter studies the susceptibility of Google's search engine to 
provide users with questionable information sources in the results for queries 
related to Dutch political parties during the Dutch provincial and European 
parliamentary elections of 2019. Our goal is to identify the presence of dubious 
sources in the results for political queries. Thus, the research questions read 
as follows. Which rankings has the search engine assigned to false and junk 
sources when querying political parties and their issues? Are there particular 
political issues and party spaces where junk news is prevalent or entirely 
absent? Do the ranks and amounts increase as the elections draw near? 

We have divided the chapter into six sections: a brief reflection about 
the methodological challenges of studying search engines, the methodology 
for building our dataset, three sets of findings, and a discussion of the 
limitations and further steps. In all it was found that junk news, specifically 
of the hyperpartisan variety, is rather pervasive in the search-demarcated 
political space, but far more so for certain actors and their issues on the far 
right of the political spectrum. 


Studying personalization, junk news, or both? 


The extent to which autosuggestions are personalized is understudied, 
but the personalization of results more generally has been the subject of 
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numerous inquiries and methodological innovations that work around 
Google's inscrutability through selectively scraping results or soliciting data 
donations. The findings from scraping batch-queried results have shown 
relatively low amounts of results affected by personalization in the Google 
search engine (Feuz et al., 2011), and the same held for Google News (Haim et 
al., 2018), suggesting that original concerns regarding the ‘filter bubble’ may 
not be as well founded any longer. Where the second method is concerned, 
Algorithm Watch, the German NGO, created a browser extension for users 
to install that regularly would make political queries, such as for [“Angela 
Merkel") (Puschmann, 2017). The results would be donated by the users 
to Algorithm Watch so as to enable a larger number of those under study 
than is normally the case. Here again the findings have shown low levels 
of personalization, but the study of the presence of certain junk sites (be 
they disinformation or another genre) could be pursued further. Another 
technique, discussed below, is to selectively scrape results ina manner that 
seeks to minimize personalization effects, thereby concentrating on the 
overall presence of junk sites rather than on whether particular users, in a 
filter bubble, are seeing more of them. 

To begin to understand the amount and placement of junk news in search 
engine space, be it around events or even after an algorithmic tweak, a query 
routine is designed, and a window of activity is chosen. (Longer-term studies 
also may be undertaken, as in the Issuedramaturg project that followed 
g/11 query results for years, but Google often changes its output formats, 
breaking automated tools (Rogers, 2013).) In order to reduce personalization, 
a research browser may be deployed, which is a clean instance of a browser 
with the user not logged in. City-level geographical personalization may be 
avoided through the use of advanced settings, choosing a particular region, 
such as the Netherlands. For projects as ours, geographical personalization 
is not viewed as a disadvantage in the sense of creating the conditions for 
a filter bubble to materialize. 

A brief mention should be made of the search engine under study. Among 
them Google is the most popular, with the largest market share of users 
in most countries, certainly in the Netherlands. As mentioned above, 
recently, the company has become entangled in the fake news debate 
through the appearance not of Russian disinformation sources (though 
that to our knowledge has not been studied in great detail), but owing 
first to the appearance of misogynistic and extremist content that the 
company previously defended as ‘reflective’ of societal concern rather than 
the product of algorithmic error or ‘culture hacking’. If one were to expand 
the number of search engines under study (to include Bing and Yahoo!, 
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for example), one could triangulate results, and inquire further into the 
normalcy and regularity of misogynistic and extremist content present in 
the top results, though one could not control for algorithmic concentration 
or the extent to which the big engines’ algorithms are anyways similar. 
The extent to which the results reflect societal concern would remain an 
open question. 


Studying Google results 


The presence of junk news within Google’s search engine results is a multi- 
causal phenomenon that may be credited to a number of factors. Among 
others, Google’s algorithm reacts and learns from users’ own consumption 
of junk sources. It is trained using varied datasets, and content producers’ 
attempt to game the search engine via search engine optimization tactics 
(Finkel et al., 2017). Given the inscrutability of Google’s tools, it is difficult to 
determine what could be causing the presence of junk sources in the Dutch 
web sphere or others. Here, rather than attempting an explanation for the 
presence of junk news, we conduct a test of the engine’s susceptibility to 
connect politically relevant queries with junk sources. 

As noted, the investigation relies on scraping as a method (Marres 
and Weltevrede, 2013), and takes as its point of departure the question of 
junk news in search returns rather than the effect of personalization in 
the creation of a filter bubble. The research seeks so-called junk news in 
search engine results, which has been defined as ‘extremist, sensationalist, 
conspiratorial and masked commentary’ (Howard et al., 2017: 1). In keeping 
with Buzzfeed News’ definition of fake news (Silverman, 2016), we also seek 
(foreign) disinformation, hyperpartisan sources as well as clickbait, which 
itself may be extreme. In order to do so we rely on a list of sources expertly 
curated by other researchers in the project (see the Appendix 6.2 in Hagen 
and Jokubauskaite, this volume). 

Generally, the research employs the ‘source distance’ approach, inquiring 
into how far from the top of the returns are the offending results (Rogers, 
2013). More specifically, we investigate how false and junk webpages are 
positioned in the first twenty Google.nl results of various queries of political 
parties and their most significant issues during the 2019 provincial elections 
campaign as well as those of the European parliamentary elections. Thus, 
this case examines the susceptibility of search engine results to junk news, 
as defined above, rather than exploring the issue of falseness and junk in 
themselves or the effectiveness of countermeasures. 
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Engine returns as political spaces 


In order to demarcate a political space in search engine results, we designed 
a query protocol based on combining the names of political parties with 
specific issues associated with their respective political agendas. In that 
sense, the underlying assumption is that junk news may have a more 
significant impact when discussed in a specific political context, such as 
election campaigns, when voters gather information to guide their choice. 
To be able to collect the results of a large number of queries, we used the 
Search Engine Scraper by the Digital Methods Initiative, a tool that allows 
one to scrape search engine results for a given query and commit them to 
a database for further scrutiny through visualization. 

The first step of the methodology consisted in constructing a list of Dutch 
parliamentary parties and locating their websites as well as Facebook pages 
(see Table 3.1). Another list was created pertaining to the parties participating 
in the European parliamentary elections. In the next step we identified the 
relevant keywords to build the queries; in the case of the Dutch provincial 
elections, these were sourced from both the party webpages and the party 
Facebook pages. For the parties competing in the European parliamentary 
elections, only the parties’ own websites were consulted. The party webpages 
provided the parties’ issue keywords. The Facebook pages furnished a more 
vernacular set of issue terms, as they contain issue keywords from users 
or citizens in the comment space. The aim of sourcing these two sets of 
keywords is to enable us to capture and compare the results for both official 
as well as more popular issue language. 

The lists of party keywords were built by collecting the platform stand- 
points (standpunten) on the party websites. There is one list for the provincial 
elections and another for the European parliamentary elections. Most 
political parties mention between five and ten keywords on their platforms, 
and all were collected. A few parties (e.g., the Staatkundig Gereformeerde 
Partij, SGP) offer longer lists which were shortened on the basis of their key 
issues. In all, the political party issue space consisted of 158 keywords across 
the thirteen parties for the provincial elections, and four keywords across 
fourteen parties for the European parliamentary elections. The vernacular 
list was made through a close reading of the comments made under the 
posts of the Facebook pages of each political party. To build this list, the 
most commented posts around the days of the elections were close-read, 
and the most representative keywords related to the views expressed by the 
commenters were chosen. Identifying the most relevant issues in the com- 
ment space on Facebook proved to be problematic, given that the comments 
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were often polarizing and emotive, especially those relating to the elections 
themselves. This citizen-enriched political issue space consisted of five 


keywords for each of the thirteen parties, making a total of 65. 


Table 3.1 List of Dutch political parties under study 


Dutch Provincial Elections European Parliamentary Elections 

Name of the Party Abbreviation Name of the Party Abbreviation 
Volkspartij voor Vrijheiden VVD Volkspartij voor Vrijheiden VVD 
Democratie Democratie 
Partij voor de Vrijheid PVV Partij voor de Vrijheid PVV 
Christen-Democratisch CDA Christen-Democratisch CDA-EV 
Appèl Appèl - Europese Volkspartij 
Democraten 66 D66 Democraten 66 D66 
GroenLinks GL GroenLinks GL 
Socialistische Partij SP ChristenUnie — Staatkundig CU - SGP 

Gereformeerde Partij 
Partij van de Arbeid PvdA Partij voor de Dieren PvdD 
ChristenUnie CU 50Plus 50plus 
Partij voor de Dieren PvdD Jezus Leeft 
50Plus 50plus Denk DENK 
Staatkundig Gereformeerde SGP Forum voor Democratie FvD 
Partij 
Denk DENK Van de Regio & Piratenpartij VR- PP 
Forum voor Democratie FvD Volt Nederland VN 

De Groenen GN 


The three lists of keywords were inputted in the Search Engine Scraper along 
with the name of each party. The results from the parties’ own websites were 
excluded. For example, for the political party D66 and the key word onderwijs 
(education), the following query was made: [onderwijs d66 -d66.nl]. Using 
the advanced search features of Google, maximum results were set to 1,000, 
and each day of our periods of interest (13-22 March and 22-24 May 2019) was 
queried separately. The date ranges included the run up to the provincial elec- 
tions on 20 March and a short election aftermath period, and the days before 
and after the European Parliamentary elections on May 23. The searches were 
conducted in a clean browser, in the Dutch Google.nl domain, in the Dutch 
language, and in the Netherlands region (through the advanced settings). The 
keyword and party were queried together so that the scraper tool delivered 
results that are related to election politics, rather than a general overview by 
querying each keyword in isolation. The keywords derived from Facebook 
were queried in the same format, using the same settings and date range. 
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The outputs of the Scraper tool are the top twenty ranked Google.nl 
results per query. The URLs in our lists were then truncated to their host 
names so that they could be cross-checked with the list of known false and 
junk websites curated by other researchers in the project. Here a formula 
was used that effectively linked the search engine results spreadsheet to 
that of the expertly curated list of junk sites. The question for each source 
concerned its ranking per query and its presence or absence in the expert 
list. All query results (per party and per official or vernacular language type) 
were marked as junk sites or not and listed in the order they were returned. 


Table 3.2 List of categories and political keywords used in the study 


Party platforms 


Foreign affairs Europese Unie, Europa, EU, Nederland en Europa, buitenland, internationale 
zaken 


Polarizing topics Islamisering, Islam 


Health Zorg, menselijke zorg 

Environment klimaat, natuur, milieu, dierenrechten, natuur en milieu 

Finances belasting, economie, inkomen, pensioen, werk en inkomen, schone 
economie, eerlijk delen, werklozen, economisch beleid, overheid en bestuur 

Safety and veiligheid, privacy, defensie, criminaliteit, rechtsstaat, terrorisme 

security 

Society waarden, gezin, respect, familie, samenleving, burgers, democratie, 
ouderen, onderwijs, goed onderwijs voor iedereen, vrijheid, verantwoorde- 
lijkheid, drugs 

Future innovatie, duurzaamheid, schone energie, energie 

Migration immigranten, migratie 

Facebook 


Foreign affairs Europa, EU, referendum 
Polarizing topics Islam, Moslim, racisme, discriminatie? 


Environment milieu, klimaat, kernenergie, energie 

Finances belasting, bezuinigingen, pensioen, onderwijs 

Society samenleving, democratie, toekomst crisis, vrouwen, vrijheid, Nederland, 
armoede 

Faith Islam, Moslim, Christendom, Christenen, geloof 

Migration migratie, immigranten, gelukszoekers, migranten 


2 Islamis placed in both faith as well as polarizing topics categories, given how it is discussed 
as shorthand for a social issue. 
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We zoomed in on those queries in which junk news showed up consistently, 
that is, for a minimum of four days within our period of interest for the 
provincial elections, and two days for the European parliamentary elections. 
The keywords that produced junk news websites in their search results in 
the first case were then grouped thematically in the following categories: 
‘foreign affairs’, ‘polarizing topics’, ‘health’, ‘environment’, ‘economy’, ‘safety 
and security’, ‘society’, ‘future’, ‘migration’ and ‘faith’, the most salient of 
which are described in some detail below (see Table 3.2). For the case of 
the European parliamentary elections, the four keywords common to all 
parties were queried: Europese Unie [“European Union", klimaat [“climate”], 
migratie [“migration”] and economie [“economy”]. 


Political parties and issue keywords 


Before analyzing the presence and positioning of junk news in Google 
web search, we would like to discuss briefly the keywords obtained for the 
provincial election campaign from the official websites and Facebook pages 
of the political parties. Comparing the composition of the categories that 
emerged from each of the two political spaces allows for showing differences 
between the matters of concern as expressed by political parties and citizens 
or social media users. Whereas political parties included keywords that 
could be grouped under the categories, ‘future’, ‘security’, and ‘health’, that 
was not the case for the Facebook users. In contrast, ‘faith’ was present in 
the Facebook comment space, whereas it was largely absent from the party 
platforms (except for the SGP, with its long list).3 There are also matters of 
concern common to citizens and political parties alike, such as ‘foreign 
affairs’, ‘economy’, ‘society’, and ‘environment’. 

Within the shared concerns there are still differences between the 
way each political space is constructed by political parties or citizens. For 
example, within the ‘foreign affairs’, ‘economy’ and ‘society’ categories, 
parties tend to refer to a wider variety of issues in comparison to the concerns 
expressed by citizens, which are mostly focused on the European Union and 
the referendum. In the economy cluster, political parties address ten issues, 
whereas citizens are concerned with far fewer. The same holds roughly for 
the society cluster. Interestingly, this trend reverses in the environment 
cluster, where users tend to express concerns about nuclear energy, while 


3 Islam was present in the party platforms (largely the PVV and SGP) but discussed in terms 
of a social issue. 
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the topic is not present in parties’ platforms (again, with the exception of 
SGP). Lastly, even though the usage of some keywords is more or less the 
same in party platforms and the Facebook comment spaces, for others it 
differed, as is the case for onderwijs (education). By the political parties 
it is framed as a societal issue, whereas in the discourse on Facebook it is 
discussed in terms of citizens’ ability to afford it. 

Visualizations are made to facilitate the analysis; they show at a glance 
the presence and ranked position of junk news for each query over time. 
The columns represent the days in the timeframes studied (13-22 March or 
22-24 May 2019). Red cells indicate the source as marked as junk news. One 
also may read the placement and distribution of it over time. 


Political party standpoint space 


The study of the political party standpoint space found overall that all false 
and junk webpages that appeared can be subcategorized as hyperpartisan. The 
one exception fell within the environment cluster in the form of a conspiracy 
website in the top twenty. Three specific websites make up for the biggest 
amount of junk: De Dagelijkse Standaard, Stop de Bankiers, and Opiniez. 
A second general observation is that queries related to right-wing parties 
returned hyperpartisan sources in a greater proportion than queries related to 
parties with other political orientations. In particular, the queries related to the 
FvD were the most populated by hyperpartisan sources. Thirdly, queries related 
to parties located at the centre of the political spectrum seem to produce 
results with less questionable sources. In those instances where hyperpartisan 
websites appear among the top twenty results for centre-oriented parties, the 
sites mainly do not make it to the top positions (though there were exceptions). 
In the political party standpoint space, most junk appears to be associated 
with queries related to keywords within ‘foreign affairs’ and ‘polarizing 
topics’. The keywords within the foreign affairs cluster mainly relate to 
the European Union and the Dutch relationship with it. Issues related 
to political parties from the centre of the political spectrum seem to be 
least connected to junk, as may be noted for 50Plus, Partij voor de Dieren 
or D66 (see Figure 3.1). Contrariwise, parties that position themselves 
strongly against the European Union are linked to a high concentration of 
junk results. For example, when looking at FvD and PVV, we can see that 
hyperpartisan sites account for 37% and 47% respectively of all the returns 
discussing the European Union. Moreover, the hyperpartisan sources are 
among the top five results throughout almost the entire time period. 
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Figure 3.1 Presence of junk news in Google.nl search engine results for political 


queries related to foreign affairs, 13-22 March 2019 
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Figure 3.2 Presence of junk news in Google.nl search engine results for political 


queries related to polarizing topics, 13-22 March 2019 
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Regarding the keywords within the ‘polarizing topics’ cluster (Figure 3.2), 
two related issues in the political party-demarcated space lead to search 
engine results with a large amount of hyperpartisan sources: Islam and 
Islamisering (Islamization). These keywords are only discussed by two 
right-wing parties, PVV and SGP, the latter from the religious right. In 
particular, when PVV is queried together with the keyword Islamisering 
(Islamization), hyperpartisan websites appear at the top of the results 
throughout the entire time span, occupying even the highest positions. 
This changes only for three days (16, 19 and 20 March), when, however, the 
amount of junk increases overall. 

The queries for environmental keywords (Figure 3.3) also lead to sig- 
nificant quantities of junk. First, it is of note that the amount decreased as 
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the elections neared, though junk websites still maintained a prominent 
position among the first 10 results. Once again, queries mentioning right-wing 
parties such as FvD are more prone to produce junk sources within their 
results. The site, ninefornews.nl, was identified here as the only conspiracy 
website in the entire data set. It emerged in the 8th position when [milieu 
FvD -www.fvd.nl] was queried, meaning that it appears on the first page 
of Google results (with default settings at 10). The centre party, D66, also 
registered significant amounts of junk for the query climate (klimaat), as 
did the centre-left labour party (PvdA). 

The cluster related to economic issues, the second largest in our set of 
results, contains nine keywords such as belasting (taxes), eerlijk delen (fair 
sharing) and werklozen (unemployed) (see Figure 3.4). Queries including 
FvD are once again among the most prone to produce hyperpartisan results, 
particularly with the keywords economie (economy), pensioen (pension) and 
belasting (taxes). When examining the remaining parties (with the possible 
exception of the PVV), the number of junk results is relatively low overall, 
and they rarely occupy the first position of the search engine results. 

The ‘society’ cluster is the largest; twenty issues were identified in this 
cluster, ranging from concerns about warden (values) to drugs (see Figure 3.5). 
Social issues produced a varied distribution of hyperpartisan sources over- 
time, with hyperpartisan websites appearing less prominently. Only three 
queries, two including the FvD and one the VVD, stand out in this cluster 
as junk-ridden: drugs, democracy and responsibility (verantwoordelijkheid). 


Lastly, the thematic cluster, ‘future developments’ (Figure 3.6), with such 
issues as duurzaamheid (sustainability), innovatie (innovation), and schone 
energie (clean energy), are addressed by parties located at the extremes of 
the political spectrum and the query results are populated by hyperpartisan 
sources. This is most evident in the case of FvD, where questionable sources 
appear nearly every day, and in four instances are returned in the top two 
positions, albeit not in close vicinity of election day, which remains largely 
unaffected by junk sources in relation to these terms. 


Vernacular political issue space 


Based on the keywords gleaned from the Facebook pages of the Dutch politi- 
cal parties, one new category was created (‘faith’) on top of the other six from 
the previous exercise. Generally, the results were similar. Queries mentioning 
right-wing parties such as PVV and FvD returned more hyperpartisan 
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Figure3.6 Presence of junk news in Google.nl search engine results for political 
queries related to future innovation, 13-22 March 2019 
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sources in the top 20 Google results, and the positions of these junk sources 
tend to be higher than in queries related to other parties. 

In the vernacular political issue space, the foreign affairs cluster received 
the most junk news returns; nearly 25% of the top 20 results are considered 
hyperpartisan websites. Migration and polarizing topics have the second 
and third highest percentage of junk websites, with 19% and 17% of junk, 
respectively. Results associated with keywords such as immigranten (im- 
migrants) and Islam were significantly populated with hyperpartisan 
sources. Issues related to environment, society, and finance obtained fewer 
junk returns (less than 10%). The faith cluster had the fewest hyperpartisan 
returns.* 

The environment cluster (which for many terms could have been merged 
with the future innovations one) is the largest one in the vernacular is- 
sue space. In it the keyword klimaat (climate) was discussed by Facebook 
users on the pages of five different parties, and milieu (environment) on 
three. For Groenlinks, D66, PvdA and VVD, the keyword klimaat features 
hyperpartisan sources in the top results (see Figure 3.7). From the election 
day onwards, fewer junk websites showed up in the top 20 results. The 
same pattern was observed in the ‘foreign affairs’ cluster (see Figure 3.8). 
Hyperpartisan websites occupied the first position for five days during our 
10-day research period for queries related to FvD and PVV. Queries for FvD 
and EU were the most likely to return junk webpages. 

Regarding the keywords grouped under the ‘polarizing topics’ cluster, 
Islam was brought up by Facebook users in the pages of four political parties: 


4 Here again, Islam is excluded, because it is considered a social issue, given the manner in 
which it is discussed in the vernacular issue space. 
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Figure3.8 Presence of junk news in Google.nl search engine results for political 
queries related to foreign affairs, using language from the Facebook 


comment space of the political parties, 13-22 March 2019 
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Figure3.9 Presence of junk news in Google.nl search engine results for political 
queries related to polarizing topics, using language from the Facebook 
comment space of the political parties, 13-22 March 2019 
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PVV, SGP, FvD and DENKS Among them, results associated with FvD received 
the most junk webpage returns in the top 20 results, and hyperpartisan 
results maintained the first position for seven days (Figure 3.9). On election 
day, however, all the parties returned few to no such results in the cluster of 
‘polarizing topics’, with the exception of the FvD. The number of junk sources 
and the rankings dropped dramatically on that day. Concerning DENK, 
although queries including it did not return many junk websites overall, 


5 Forthe political party, DENK, Islam, while discussed as a social issue, also could be catego- 
rized as ‘faith’, 
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Figure 3.10 Presence of junk news in Google.nl search engine results for political 
queries related to migration, using language from the Facebook 


comment space of the political parties, 13-22 March 2019 
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they were found in relation to the issues Moslim (Muslim), discriminatie 
(discrimination) and racisme (racism). 

The migration cluster (Figure 3.10) was more prominent in the discussion 
space on Facebook pages than in the platforms of the political parties. 
Keywords as migranten (migrants), migratie (migration), immigranten 
(immigrants) and gelukszoekers (fortune seekers) were often mentioned in 
the comments on the Facebook homepages of political parties. (Gelukszoekers 
could be said to be a pejorative term for economic migrants.) Results for 
three parties in particular (FvD, PVV and VVD) had junk webpages in the 
top 10 Google search results, and they were spread evenly over the ten-day 
research period. It is also noteworthy to see that there appears to be a 
decreasing tendency of junk news from the election day onwards in this 
cluster, both in terms of the amount as well as the rankings. 


European Parliamentary Elections 


The results for the political party standpoint space during the European 
Parliamentary elections also show a consistent presence of junk news. 
Three major findings are worth mentioning. Firstly, as figures 11 and 12 
make evident, the presence of junk sources in the Google search engine 
results was lower during the European parliamentary elections than during 
the provincial elections. In many cases, our queries combining issues with 
parties did not produce links to hyperpartisan material or only did so for 
one day. Only 25% of our queries returned junk for more than one day 
during the three-day period under research. Of the four keywords queried, 
the one that produced the least amount of junk in combination with party 
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names was economie (economy). In contrast, migratie was the most prone 
to returning junk, with queries related to five parties consistently returning 
junk websites between May 22 and 24 (see Figure 3.11). 

Queries in combination with the political party DENK were particularly 
junk ridden. On the day of the election and the day after, almost half of the 
results provided by the Google search engine is problematic. It is particularly 
relevant that during the 24* of May, hyperpartisan websites occupied the 
top seven positions. 

A second relevant finding is that the presence of hyperpartisan resources 
is more prevalent on the day after the election than the day before, in contrast 
to what happened during the provincial elections. Although this is the 
case for all parties and keywords where junk sources were identified, it 
was especially prominent for the issues of migration, economy and climate 
(keywords migratie, economie and klimaat). In most cases, junk sources 
also occupy the first positions in the results during the 24" of May. We 
cannot answer the question of whether this phenomenon can be credited 
to the Google search algorithms reacting to an increase in searches related 
to the elections, to a surge in the activity of hyperpartisan websites after 
they took place, or to some combination. A close reading of the results 
in the highest positions, however, shows that the hyperpartisan sources 
behave as one would expect from any information provider during election 
season, if only keeping their radical tone; before the election they provide 
predictions about the results, on the day of the election they invite citizens 
to vote, and on the day after they discuss the results. An article by the De 
Dagelijkse Standaard that consistently occupied the top result for various 
keywords and parties is a reflection about how the FvD and leader Thierry 
Baudet need to tone down their radical discourse in order to become a more 
powerful political force. 

Lastly, whereas during the Dutch provincial elections the queries related 
to right-wing parties were more strongly connected by the search engine 
with junk sources, during the European elections this is not the case. For 
example, although queries performed in May involving FvD also produced 
results pointing to hyperpartisan websites — similarly to the results obtained 
in March — it was those related to DENK which, in aggregate terms, produced 
more junk (i.e., 25 for FvD and 27 for DENK). However, the case of DENK 
is difficult to assess given that, apart from the name of a party, it is also a 
common Dutch word (‘think’, in English). Although this does not change 
the fact that people looking for information about this party would likely be 
exposed to junk sources, the content may not specifically relate to DENK. In 
fact, the highest-ranking result for the query [migratie DENK -bewegingdenk. 
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Figure 3.11 Presence of junk news in Google.nl search engine results for political 
queries related to migration and European Union issues, 22-24 May 2019 
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Figure 3.12 Presence of junk news in Google.nl search engine results for political 
queries related to climate and economic issues, 22-24 May 2019 
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nl] is an article in the hyperpartisan website De Dagelijkse Standaard that 
makes no specific mention to this party, but instead generally reflects on 
the results of the elections and specifically on the demise of the PVV. 


Conclusions: Junk news in search engine results 


The goal of this research is to locate the presence and ranking of junk 
websites within the first twenty Google.nl results of queries concerning 
Dutch political parties and their most significant issues during the 2019 
provincial and European parliamentary elections. The keyword queries 
were built by combining political party names with keywords retrieved from 
political party platforms and party Facebook page comment spaces (in the 
case of the provincial elections). We clustered the keywords into categories to 
enable a comparative analysis. At the outset the research questions were as 
follows: Which rankings has the search engine assigned to junk sources when 
querying political parties and their issues? Are there particular political 
issues and party spaces where junk news is prevalent or largely absent? Do 
the ranks and amounts increase as the elections draw near? 

Our results indicate that the junk websites present in the results of our 
Google.nl political queries are almost exclusively hyperpartisan, rather than 
sources spreading disinformation, conspiracy theories, or clickbait. Three 
websites, namely De Dagelijkse Standaard, Stop de Bankiers and Opiniez, 
account for the largest portion of the junk sources identified. We did not find 
fake advocacy groups or foreign disinformation operatives. Furthermore, we 
found that queries involving right-wing parties were more prone to result in 
exposure to hyperpartisan sources than those associated with centre-left or 
left-wing parties. For most keywords, hyperpartisan websites appeared in 
the top positions, and certainly always within the first page of results. Our 
findings suggest that on Google.nl there is a considerably high probability 
that junk news is outputted on the first page of results when the queries 
concern right-wing parties and their issues. 

Another finding is that hyperpartisan results spread unevenly during 
our research period. For the case of the Dutch provincial elections, they are 
more present before the day of the election and drop their presence and 
ranking, in some cases dramatically, on election day and in the immediate 
aftermath, for instance in the case of migration issues in the vernacular 
issue space. In contrast, during the European parliamentary elections this 
phenomenon reversed, and junk sources were more prominent the day 
after the elections. 
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Concerning the comparison between the two spaces, the vernacular space 
had the highest percentage of junk news returns, especially in the categories 
of foreign affairs and migration. In the political party standpoint space, the 
differences among themes is smaller. In other words, the queries designed 
with the political language of Dutch Facebook users were more likely to 
result in hyperpartisan results than the queries built with the standpoint 
language of political parties. Although more research is needed in this 
regard, such a finding suggests that the discourse of normal citizens, or 
those commenting on party Facebook pages, is more politically contentious 
than that of political parties. 

The results are indicative of the amount of hyperpartisan material in 
political space in Google.nl rather than conclusive, for they derive from a 
particular query strategy and not from multiple strategies and are only a 
snapshot from a particular event-related timeframe. The data set we built 
could also be read more closely, and additional junk sources could be found, 
meaning that we could have undercounted (rather than overcounted). 

Given that our intention is to determine the susceptibility of the Google 
search engine to junk news, the question remains whether the location of the 
results of politically charged queries can be credited to an optimization effort 
on the side of hyperpartisan content generators, an overall susceptibility 
of Google’s search algorithm to provide questionable content to its users, 
consumers’ preference for low quality information, or some combination 
of the three. 

While discussing the two latter hypotheses would require more space, a 
few words can be said about the first. A possible way to detect search engine 
optimization strategies consists of using one of the many online services 
providing SEO analysis. Given the proprietary nature of their methodologies, 
however, the results should be interpreted with caution. We submitted 
the three most recurrent junk websites we found during our research to 
the service SEO Tester Online, a tool that measures a website’s readiness to 
achieve top positions within search results. This tool provides analysis in 
four different categories: basic (related to the overall online presence of 
the website), content (measuring the richness of the keywords that trig- 
ger the website to pop up in search results), web performance (indicating 
how fast the website can be rendered in mobile and desktop devices), and 
social (providing information about the website’s engagement with users 
through social networks). De Dagelijkse Standaard obtained a score of 
56/100, faring the lowest in their web performance, and the highest in its 
social engagement. Opiniez obtained a score of 62/100, faring the best in web 
performance and the lowest in content, although it obtained an excellent 
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score in the number of unique keywords. Stop de bankiers obtained the 
highest score, with 74/100, including a 100/100 in the assessment of keywords 
which may lead to the site through search engines. It also fared the best in 
social engagement and the lowest in web performance. 

The reports produced by SEO Tester Online, which for all websites included 
a considerable amount of suggestions about how to improve the websites’ 
code, suggest that, at least currently, content producers are not actively 
seeking to artificially alter the performance of their sites. Consequently, 
the presence of these junk sources in our analysis is more likely related 
to an inherent susceptibility of Google’s search engine and/or an organic 
result of users’ preferences. 

Further research is necessary to paint a clearer picture regarding the 
increased consumption of junk news. One could repeat the work for longer 
periods of time in order to ascertain the extent to which the presence of 
these or other dubious source types is becoming more widespread or even 
disappearing in the top results for political queries, as is currently the case 
with clickbait. Furthermore, specific sources could be monitored over time 
to track the performance of their content (and the changes to their code), 
in order to detect attempts to artificially increase relevance and thus the 
ranking assigned by search engine algorithms. 
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4 Thecirculation of political news on 
Twitter during the Dutch elections 


Sabine Niederer and Maarten Groen’ 


Abstract 

This chapter enquires into the resonance of junk news on Twitter during 
the campaign periods prior to the 2019 Dutch Provincial elections and 
European Parliamentary elections. Querying Twitter for political topics 
related to the two elections, and various divisive social issues such as 
Zwarte Piet and MH17, we analyse the spread and prominence of prob- 
lematic sources. We also examined the claim that Twitter is susceptible 
to abuse by bot and troll-like users, and found that troll-like users were 
active across all political and issue spaces during the Dutch Provincial 
elections of 2019. Divisive issues remain steadily (even if marginally) 
active in junk and tendentious news throughout the tested time frames, 
suggesting these issues are year-round rather than event-based or seasonal, 
as they are in mainstream media. 


Keywords: Twitter, social media analysis, junk news, social issues, trolling, 
digital methods 


Introduction 


In 2018 the Dutch daily newspaper de Volkskrant published an article en- 
titled ‘The troll army of pop artist Dotan’, which revealed how the Dutch 
singer-songwriter had made use of fictitious accounts pretending to be 
fans (Misérus and van der Noordaa, 2018a). The fake fans were highly active 


1 Theresearch reported here was undertaken in collaboration with Laval Boulos, Peter Fussy, 
Oana Patrici, Maria Stenzel Timmermans, Emile den Tex, Carlo De Gaetano, and Federica 
Bardelli. 
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across social media platforms (including Facebook, Twitter, and Instagram) 
where they circulated heart-warming stories about the artist, requested 
his songs on Dutch and German radio stations, and actively tried to edit 
the Wikipedia pages about the artist and his mother (who is also a Dutch 
singer). At the root of these activities were 140 accounts that the newspaper 
retrieved, at least one of which connected directly to the artist’s own Gmail 
account, and others to accomplices. Dotan’s case is perhaps the most-known 
example of artificially boosted accounts and content in the Netherlands, 
but certainly not the only known case of such behaviour. The politicians 
Geert Wilders and members of the political party Denk were found to have 
suspiciously inflated follower counts, which surfaced when Twitter started 
deleting unvalidated users (NOS, 2018). 

The present study builds on previous digital research in which the social 
media platform Twitter, used by over 326 million monthly active users 
accounting for 500 million tweets per day, is repurposed for social research 
(Omnicore, 2019). As with Dotan and the Dutch politicians mentioned above, 
it similarly looks into social media use and the question of manipulation, 
in particular in political spaces around elections. It studies troll-like and 
artificial boosting as well as the circulation of junk and tendentious news 
sources during two election campaign periods in 2019. Initially intending to 
detect the possible presence of Russian disinformation in the Dutch Twitter 
space, the study enquires into coordinated campaigning around divisive 
issues and ascertains the extent of homegrown junk news in Dutch political 
Twitter, including hyperpartisan, conspiracy and clickbait sources. So-called 
(and self-identified) tendentious sources such as Geenstijl.nl and TPO.nl are 
labelled as such, and one could argue that they are mainstreaming, given 
how they are shared, as we discuss below. These two sources are part of the 
‘anti-establishment established source’ set, and as such are closely related 
to an emerging alternative media landscape (see Tuters, this volume). 

In employing digital research methods and techniques, the analysis makes 
use of the platform's own features and cultures of use, which offer built-in 
structuring of the content being shared (Rogers, 2019). These are repurposed 
for social or political research. Hashtags can be repurposed as content 
categories or issue activity indicators, retweeting suggests ‘pass-along value’, 
and the @reply and @mention functionalities network users and their 
content to fellow users and content (Niederer, 2018). Through an analysis of 
@replies, Twitter can be studied ‘as a conversation-maker, where one may 
explore the extent to which there is dialogue, or broadcasting’ (Honeycutt 
and Herring, 2009; boyd et al., 2010). The @mentions may contribute to the 
inquiry of dominant voice — certain understandings of issues can be shaped 
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by the actors most mentioned in a tweet corpus, and also by the actors that 
are the most vocal. Twitter can be studied as a social network of professional 
information-sharers (Java, 2007). It also can be considered a ‘rebroadcaster’ 
of (political) news, in which the platform’s built-in algorithms reinforce the 
issues and framings discussed there as so-called trending topics (Kwak et 
al., 2010). Furthermore, Twitter is often moving information faster than the 
news, and Twitter content in some cases becomes news (Niederer, 2018). As 
news and mass media sources strive to make their content ‘platform-ready’ 
(Helmond, 2015), political news, other mass media content and new platforms 
become further entangled, forming a hybrid media system (Chadwick, 2013). 
Here, professional journalists include tweets in their stories, and when their 
work has been published, they may post a link to that article on Twitter and 
other social media, using the platforms both as a source of information and 
as a channel for the distribution of their own work. 

Critiques of digital social research take issue with its dependency on 
the already problematic hegemony of proprietary social media platforms. 
On a methodological level, scholars warn of the sheer impossibility of 
distinguishing between the working logic of web platforms and exemplary 
‘platform artefacts’ (Marres, 2015; Marres and Weltevrede, 2013; Rogers, 2013; 
Niederer, 2019). How do we know whether the most-retweeted Twitter post 
is the most relevant, or the most Twitter-friendly (Marres, 2015)? One way 
to approach this issue is to take into account the socio-technical specifics 
of each platform, and to regard Twitter and other social media platforms 
as distinct windows on an issue. Rather than questioning the relevance of 
the platform for the elections, we then ask: how does Twitter present the 
elections? And how does this compare to how other social media platforms 
cover the topic? Such lines of questioning open up avenues for qualitative 
and empirical digital research across political events and social issues as 
they resonate online and offer insights into the cultures of use of the various 
platforms. In this present study, Twitter can be seen to produce political 
subspaces around divisive issues, in which a relatively small number of 
highly active, troll-like users sow division and where junk news at times 
outperforms mainstream news. 


Troll-like user activity during the 2017 Dutch general elections 
The present study follows from an earlier one, which itself concerned Dutch 


elections. In the lead up to the 2017 Dutch general elections for the national 
parliament, journalists revealed the use of sock puppets (De, false online 
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Figure4.1 Political party leaders as trolling targets on Twitter during the 2o17 
Dutch general elections. Each dot represents one mention (by a user 
mentioning political leaders at least 100 times). Red represents an 
attack, and green represents a favourable mention. 


@MinPres (VVD) @geertwilderspvv (PVV) Bthierrybaudet (Forum voor Democratie) 


Source: Borra et al., 2017 


identities assumed to deceive and influence opinion) by the political party 
Denk, in order to amplify their online messages and attack their political 
opponents on Twitter and Facebook (Kouwenhoven and Logtenberg, 2017). 
In an empirical study as part of the Field Guide to Fake News (Bounegru et 
al., 2018), we studied troll-like behaviour in Twitter, developing a research 
protocol for identifying and analyzing political trolling, which in this case 
referred to repeated attacks of politicians on Twitter. It focused on the 
sources of troll-like activity De, which user accounts target politicians?), 
their targets (who do these troll-like users address?), and the characteristics 
of these practices (what do troll-like users do?) (Borra et al., 2018). 

The detection of user accounts engaging in political trolling behaviour 
starts by compiling a list of potential targets. The aforementioned study 
looked into the user accounts of 28 political party leaders participating 
in the 2017 elections. The users that @-mention them were queried. For 
the most-active users per @mention, their posts in which they @-mention 
the political leaders were qualitatively studied. In a next step, only those 
who @-mention one or more political leaders at least 100 times during 
a one-month period (8 February-8 March 2017) were retained, and their 
tweets coded for being favourable or unfavourable of the politician. The 
study found an asymmetry in the troll-like behaviour across the political 
spectrum, as more left-wing politicians were being targeted by negative 
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mentions while most right-wing politicians were receiving support (see 
Figure 4.1). There are exceptions, such as Emile Roemer (SP) and Marianne 
Thieme (PvdD), who in this time frame received only support by troll-like 
users, and Prime Minister Mark Rutte (VVD) who received unfavourable 
mentions, in particular on his personal account though less so on his official 
@MinPres account. 

To classify the sources of political trolling, we used the same list of 24 
highly active and troll-like users (mentioning political leaders at least 100 
times in the one-month time frame), and collected their profile informa- 
tion (description, profile picture and banner) from the Twitter interface. 
If the profiles had a profile picture, Google reverse image search was used 
to check these images for authenticity. Then, using the Twitter API, the 
creation date for each of these accounts was retrieved, in order to assess 
whether accounts in our dataset had been created on the same date. This 
analysis provided a more nuanced view of the user accounts responsible for 
the trolling behaviour. Of the 24 accounts still active at the time of study, 
three users appeared to be sock puppets created for trolling activities. They 
had very similar profiles and had been created within a short timeframe. 
Another six accounts in the data set promoted the same anti-Islam agenda, 
but were not determined to be fake accounts. 

To characterize the substance of the trolling practices, the study looked 
at the issues and the media sources that resonate in the set of tweets. To 
identify the issues, the hashtags used by the highly active and trolling 
users in their tweets (that @mention a political leader) were collected and 
analyzed. Most tweets that include hashtags were found to mention the 
right-wing populist candidate Geert Wilders, and most hashtags referred 
to the issues in PVV’s political messages from 2017 (‘Nexit’, ‘StopIslam’ and 
‘BanIslam’), as well as those pertaining to expressions of Dutch patriotism 
(Borra et al., 2017: 188). To assess which media sources were circulated by 
the troll-like users, the most-circulated URLs in the tweets were collected 
and categorized. For the 2017 general elections, the most-tweeted media 
sources by the 24 trolling users were the Dutch extreme blog fenixx.org 
followed by the anti-Islam site Jihad Watch and the right-wing think tank 
the Gatestone Institute (Borra et al., 2018: 192). 


Research questions and data collection 


For the study presented in the next section, the main research question is 
to what extent junk news sources and troll-like user accounts are present 
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Table 4.1 Query overview showing the election campaign period (Provincial, EU 
or both), the political or issue space and the query made resulting in 


Twitter data sets 


Elections Topic Query 
PS General Ps2019, Ps19, verkiezingen 
EU General EUverkiezingen2019, euverkiezingen, ep2019, eu2019, 


euelections2019, verkiezingen, verkiezingen2019, EU, Europa, 
Europese Unie, europeseverkiezingen 

PS Partyleaders Mark Rutte, MinPres, markrutte, Geert Wilders, geertwilder- 
spvv, Thierry Baudet, thierrybaudet, Jesse Klaver, jesseklaver, 
Rob Jetten, RobJetten, Lilian Marijnissen, MarijnissenL, Mari- 
anne Thieme, mariannethieme, Gert-Jan Segers, gertjansegers, 
Lodewijk Asscher, LodewijkA, Tunahan Kuzu, tunahankuzu, 
Henk Krol, HenkKrol, Klaas-Jan Dijkhoff, dijkhoff, Sybrand Buma, 
sybrandbuma,Kees van der Staaij, keesvdstaaij 

EU Party leaders SophieintVeld, Esther_de_Lange, mjrldegraaff, malikazmani, 
arnouthoekstra, TimmermansEU, petervdalen, BasEickhout, 
anjahazekamp, ToineManders, florens0148, atonca, paulbeasd, 
djeppink, sentwierda, RLanschot, MinPres, markrutte, geertwil- 
derspvv, thierrybaudet, jesseklaver, RobJetten, MarijnissenL, 
mariannethieme, gertjansegers, LodewijkA, tunahankuzu, 
HenkKrol, dijkhoff, sybrandbuma, keesvdstaaij 


PSandEU MH17 mh17 

PSandEU ZwartePiet Zwartepiet, zwarte piet 

PS and EU Climate klimaat 

PSandEU Fake news Fake news, fakenews, nepnieuws, desinformatie, junknieuws 
PS Utrecht utrecht, 24oktoberplein, gokmen tanis, gokman tanis 


on Twitter around both the provincial and the European parliamentary 
elections in the Netherlands in 2019. To answer these questions, we examine 
Twitter activity concerning the elections, the party leadership as well as 
political candidates, and zoom in on potentially divisive issues, including 
Zwarte Piet and MH17. 

In addressing these research questions, queries are formulated to de- 
marcate the political and issue spaces in Twitter (see Table 4.1). The data 
are collected using the commercial social media monitoring tool, Coosto, 
and the Twitter Capturing and Analysis Toolkit developed by the Digital 
Methods Initiative (DMI-TCAT). Coosto was used to retrieve data from 
both the provincial and European election periods, in order to conduct 
a comparative analysis of the engagement with mainstream and junk 
news across political and issue spaces, and the presence of troll-like users 
in these spaces, as discussed in detail in the next sections. DMI-TCAT, a 
tool that ‘provides robust and reproducible data capture and analysis and 
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Figure 4.2 Engagement of mainstream (blue) and junk news (pink) articles 
during the Dutch Provincial election campaign (left) and the European 


Election campaign period (right) 
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Line graphs; visualizations by Federica Bardelli 


interlinks with existing analytical software' (Borra and Rieder, 2o14: 262), 
was used to analyze the engagement with junk and tendentious news sources 
and the users responsible for this engagement. While some collections 
(or ‘bins,’ in the terminology of the TCAT-tool) were created only for this 
study, others had been running for months prior, such as MH17, or in the 
case of Zwarte Piet even years (with a bin that was created in December 
of 2o17). The set for the Utrecht tram shooting was created on the day that 
event took place, 18 March 2019. For this study, the sets were limited to the 
provincial elections campaign period (18 February-25 March 2019) and the 
European Parliamentary election campaign period (26 April-24 May). The 
one exception was the Utrecht tram shooting set, which was only included 
in the Provincial Elections campaign period, as it took place during that 
time frame. 


Junk news sources and troll-like users during the provincial 
elections on Twitter 


During both the provincial and the European election campaigns we tracked 
the resonance of mainstream, junk and tendentious sources in Twitter. We 
did so around the potentially divisive issues of Zwarte Piet and MH17 and 
chose to include climate and fake news (as an issue). Furthermore, we tracked 
the resonance of news sources for the political spaces of the (Provincial 
and EU) elections, as well as the party leadership and political candidates. 
For each of the elections, we demarcated a five-week campaign period. Per 
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Figure 4.3 Engagement with mainstream news (blue) and junk news (pink) for 
the issue of Zwarte Piet (top left) and MH17 (top right) and during the 
Provincial elections, and the EU elections (bottom left and right) 
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Line graphs; visualization by Federica Bardelli 


political and issue space, and for each of the five weeks of the campaign, 
the most-shared links (up to a maximum of 500) were collected and coded 
(for mainstream or junk news of various types, using the aforementioned 
expert list). The engagement scores for the mainstream and junk news source 
engagement per week were visualized as line graphs, as in the well-known 
Buzzfeed News study (Silverman, 2016). 

For both election campaign periods, overall the mainstream news 
outperforms junk news (see Figure 4.2). When zooming in on the political 
spaces of the elections and the party leadership and political candidates, 
the mainstream news sources garner far more engagement than junk news. 
A look at the top 500 most engaged-with links shows the rise and fall of 
mainstream hosts circulated in the issue space, and the relatively small but 
steady resonance of junk news hosts, which during the provincial election 
campaign rises slightly in its last week. 
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Divisive issues: Zwarte Piet and MH17 


Both for Zwarte Piet and MH17, there are instances in which junk news 
outperforms mainstream news. In the climate and fake news datasets 
mainstream news outperforms junk news in all weeks. The line graphs 
in Figure 4.3 include a zoomed-in view that renders visible the moments 
in which junk news is more engaged with than the mainstream news. 
For the controversial topic of Zwarte Piet, during the Provincial election 
period mainstream news receives more engagement. Junk news outperforms 
mainstream news in weeks three and four of the European parliamentary 
elections campaign. The article mostly responsible for this peak in week 
three is a short commentary on tendentious-hyperpartisan website The Post 
Online, about the proposal by Dutch politician Sylvana Simons (addressed 
to the Amsterdam Municipality) to ban the ‘racist caricature of Zwarte 
Piet’ in the city of Amsterdam. When one removes The Post Online from 
the graph, the results remain the same apart from the one week in May 
during the European parliamentary election period where now mainstream 
news outperforms junk (see Appendix 4.1). In week four, an article on 
Cultuurondervuur.nu (‘culture under fire’) entitled ‘Jerry Afriyie receives 
funding for anti-Zwarte Piet educational materials’ is responsible for the 
increased activity. In it, activist Jerry Afriyie is described as a ‘Zwarte Piet 
hater’ (cultuurondervuur.nu, 2019). 

For the issue of MH17, during the Provincial elections campaign there 
are times in which junk news outperforms mainstream news in terms of 
engagement. For the European parliamentary elections, the mainstream 
attracts more engagement, but during certain periods junk news is on a 
similar level as the mainstream. The peaks that occur during the Pro- 
vincial elections are mainly caused by engagement with a piece from 
citizen-journalist Max van der Werff, on his website kremlintroll.nl, in 
which he demands rectification of an article in De Groene Amsterdammer 
(from August 2018) about Russian internet trolls (van der Werff, 2019). Two 
other articles that attract engagement are from the hyperpartisan website 
jdreport.nl, questioning the integrity of the MH17 investigation, and in 
one Frans Timmermans (who would win a seat for the PvdA in the EU 
parliamentary elections) is named as part of an ‘MH17-doofpot’, or cover-up 
(jdreport.nl, 2019). In week four of the Provincial elections campaign 
period, the Kremlintroll piece requesting rectification is particularly 
actively shared. Simultaneously, the interlinked article with the actual 
critiques of the article from De Groene Amsterdammer is receiving more 
engagement. 
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During the EU election campaign, mainstream news receives more 
engagement. It is important to note, however, that aside from a peak in 
mainstream news in week three of the campaign, its engagement level is 
equal to that of junk news sources. Where in the mainstream certain events 
cause peaks in media coverage, it appears that for junk news these divisive 
issues are continuous and year-round. Zwarte Piet may not be a subject 
matter in the mainstream news in Springtime, but it remains a matter of 
concern and a source of engagement in junk news media. 


Troll-like users during the Dutch provincial and European 
elections on Twitter 


For the Dutch provincial elections campaign period, the next step in the 
study is to look closely at the user activity related to the Dutch provincial 
elections and the political party leadership, as well as coverage of the 
potentially divisive issues of Zwarte Piet, MH17 and the Utrecht tram 
shooting. As a first step, the URLs (hosts) were extracted from the sets of 
tweets and checked against a collaboratively compiled expert list of junk 
and tendentious news sources. Similarly, the users active in each of the sets 
of tweets were checked against a list of flagged users. Here, we made use 
of existing lists from the previous project in The Field Guide to Fake News 
(Borra et al., 2017) and expanded these lists. To do so, we extracted top 
users from the data sets of Zwarte Piet, MH17, Utrecht tram shooting, the 
Dutch provincial elections and the political party leadership and followed 
a protocol adapted from the aforementioned study, and combined them 
with research on credibility metrics (Borra et al., 2017; Groot et al., 2019).” 
With the Compare List tool (Borra, 2013), the study assessed whether any 
of the flagged users were active in one or more of the political issue spaces. 
Zwarte Piet had an initial list of 26 potentially troll- or bot-like accounts, 


2 For this particular study, to identify potentially troll-like users in the data sets, the top 15 
most-active users in the set were selected, as well as the top 15 users who were highly active yet 
at the same time very low on visibility (i.e., rarely or not at all @mentioned). Then, the profiles 
of these user accounts were checked for the following flags: mostly retweeting, or retweeting 
in several languages (as possible indicators of automation) which is of interest given the wide 
distribution of easily acquirable retweet bots (McGarry, 2013); profile oddities such as inauthentic 
user's profile images, which were checked with Google Image search to assess their authenticity; 
a recently created account; a high following count (of over 1,000); a username with over 3 numbers 
in it; high tweet frequency as tweeting over 200 times mentioning the issue; posting 20 tweets 
or more times per day; and, whether the user seems to mostly retweet more often rather than 
tweet his/her own content. 
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five of which had already been taken offline at the time of inquiry. Of 
the 21 remaining each was flagged as potentially troll-like; one of which 
described itself as a retweet bot (in the user profile). For MH17, of an initial 
list of 26 potentially troll- or bot-like user accounts, two were inactive at the 
time of inquiry. Of the remaining user accounts, 13 accounts were flagged, 
and 10 were not. For the Utrecht tram shooting, from an initial list of 23 
potential troll- or bot-like accounts, 10 were flagged after examination. For 
the provincial elections dataset, the list of potentially troll- or bot-like users 
entailed 24 accounts, 17 of which were flagged according to our criteria and 
one of them described itself as a bot. 

Subsequently, these flagged users were checked for activity in more than 
one issue. This would make sense for those data sets that are of related 
topics, such as the provincial elections and the political leadership. When 
users are active across distinct controversial issues such as Zwarte Piet, 
Miz, and the shooting in Utrecht, which have in common their potential 
divisiveness, such multi-issue users and the content they circulate would 
be further scrutinized. In fact, 14 flagged accounts are common to all of 
the five political issue spaces, and as many as 29 flagged user accounts are 
common to four of the data sets, pointing to efforts to fuel division during 
the election period. 

Figure 4.4 provides an overview of the tweet- and user counts per issue, as 
well as the most-resonating hashtags, and most-retweeted tweets, during the 
time around the elections (18 February-25 March 2019). The analysis shows 
that there is no disinformation resonating in the top 10 hosts per political and 
issue space. The top hosts are mostly (Dutch and international) mainstream 
news media. The hyperpartisan site Opiniez.nl is among the top 10 hosts 
for Zwarte Piet in the provincial elections space, and the tendentious site 
geenstijl.nl is shared for MH17 and PS2019. Junk sources are present across 
political and issue spaces around MH17, Zwarte Piet, Utrecht, PS2019, and the 
Dutch party leadership. There are junk news hosts that are common across 
all five issues: Ninefornews.nl, fenixx.org, tref.eu, ejbron.wordpress.com, 
drimble.nl (a particular story), and dagelijksestandaard.nl. Hyperpartisan 
and conspiracy sources are mostly circulated by flagged users. However, 
some hyperpartisan and tendentious sources are being mainstreamed, and 
circulated by regular (as in: unflagged) users. These include tendentious- 
hyperpartisan host The Post Online and hyperpartisan sources, De Dagelijkse 
Standaard and Fenixx. 

Looking at the time frame around the provincial elections, flagged users 
are among the top, most active users across issues. In particular for Zwarte 
Piet and MH17, six of the top ten users are flagged accounts. Analyzing the 


134 


SABINE NIEDERER AND MAARTEN GROEN 


Figure4.4 Tweetand user counts, top hashtags, and most-retweeted tweets 


during the Dutch provincial election period of 2019 
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top @-mentioned users in tweets about Zwarte Piet and MH17, we found that 
two flagged user accounts are among the top 10 @mentioned. When analyz- 
ing the most-used hashtags across the issues, what stands out is that the top 
hashtags used in the MH17 issue space all seem to be Pro-Russian. Across 
the issue spaces of Zwarte Piet, MH17 and PS2019, we see the resonance 
of right-wing political party hashtags, such as PVV and FvD. Zwarte Piet 
contains hashtags both for pro-Zwarte Piet (e.g., ‘blokkeerfriezen’, referring to 
the Frisian counter-protest in Dokkum against anti-Zwarte Piet protesters of 
‘Kick out Zwarte Piet’, which can be found in the data set with hashtag #kozp, 
in which they blocked the highway to prevent anti-Zwarte Piet protesters 
from entering their town) and anti-Zwarte Piet, e.g., ‘SamenTegenRacisme’, 
which translates as ‘united against racism’. 

For the EU election campaigns, we similarly investigated the activity 
of flagged users in the political and issue space. For the political spaces, 
the top 1000 most active users were collected for the general EU election 
hashtags and the political leaders relevant to the EU election campaigns. 
For the issue spaces, the top 1000 most active users were collected on the 
topics of climate change, Zwarte Piet, MH17 and fake news. These lists of 
top users were matched with the flagged users list from the first part of 
the empirical study. Because some topics were more active than others, the 
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activity of the top 1000 users varies per dataset. For the more generic EU 
set, the top 1000 users each posted more than 44 tweets in the EU election 
period. In comparison, in the Zwarte Piet dataset the top 1000 users each 
posted two or more tweets. 

Of the flagged users list, eight users were active in all six issue spaces 
during the EU campaign period. Three users were active in five of the 
spaces and another three users in four of the spaces. Four of the eight 
users active in all spaces were also active in all the provincial election 
period datasets. From the users active in all datasets, the top user posted 
2,781 tweets. 2,578 of those tweets were in the general EU and party 
leader dataset. This user is not only retweeting other content, but also 
posts his own content. The content in the EU Elections period can be 
characterized as anti-EU, anti-immigration, pro-PVV/FvD and critical 
of all other parties. 


The circulation of junk and tendentious news during the 
provincial elections 


To gain a better view of these troll-like, junk and tendentious news activities, 
a next step zooms in on the circulation of these news sources during the 
campaign period in each of the political issue spaces. Visualized as network 
graphs, the analysis considers whether such news sources are circulated by 
flagged or regular (non-flagged) users.3 Each host-user bi-partite network 
graph includes a short overview of the user and host types per data set, 
clearly illustrating that the number of flagged users and the circulation 
of junk or tendentious news sources are outnumbered by unflagged users 
and the circulation of mainstream news. Thus, these visualizations should 
be read as a zoom-in on a particular, small set of hosts that are of interest 
to the study of the presence and circulation of junk news and tendentious 
news and the users that circulate them. 

In each issue space, hyperpartisan sources are circulated the most. And 
while the issue space of Zwarte Piet is dominated by the circulation of 
hyperpartisan sources being shared by flagged but also by regular users, 
the main junk news sources for MH17 are more diverse in composition. 
Here, we see a mix of tendentious, hyperpartisan, as well as conspiracy 
hosts. For the Utrecht shooting, tendentious and hyperpartisan hosts are 
circulated the most, by flagged and regular users, making them appear as 


3 Regular in this case in fact strictly speaking means not flagged. 
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Figure 4.5 Gephi visualization of Zwarte Piet host-user network during the 
provincial elections campaign period, depicting only junk and 


tendentious hosts and the user accounts that circulate these sources 
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mainstream. The junk news and tendentious sources in both of the political 
spaces, PS2019 and the party leaders, revolve around mostly hyperpartisan 
and tendentious sources. 

The host-user network of the Zwarte Piet issue space (Figure 4.5) is dense 
and, as said, is dominated by the circulation ofhyperpartisan sources such 
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Figure4.6 Gephi visualization of MH17 host-user network during the provincial 
elections campaign period, depicting only junk and tendentious hosts 


and the user accounts that circulate these sources 
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as dagelijksestandaard.nl, fenixx.org, cultuurondervuur.nu and opiniez. 
nl, and, at a slightly lower level, the tendentious source geenstijl.nl. These 
central nodes are the sources of choice for the majority of the flagged users, 
but also have been shared by regular users, who demonstrate a preference 
for the hyperpartisan source, dagelijksestandaard.nl. One clickbait host 
(tpook.nl), which can be found in the outskirts of the graph, stands out as 
being circulated by both flagged and regular users. 

The network visualization of the MH17 junk news source circulation 
(Figure 4.6) shows a different source composition to that of Zwarte Piet, 
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Figure4.7 Gephi visualization of Utrecht shooting host-user network during 
the provincial elections campaign period, depicting only junk and 


tendentious hosts and the user accounts that circulate these sources 
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which had hyperpartisan sources at its core. For MH17, we see a more di- 
verse set of sources central to the network: tendentious source geenstijl.nl, 
hyperpartisan/conspiracy source novini.nl, and a set of two other conspiracy 
hosts (ninefornews.nl and niguru.co), which have been widely circulated 
by flagged users. 

The flagged users in this issue space mostly circulate tendentious hosts, 
such as geenstijl.nl, and hyperpartisan and conspiracy sites, hersteldere- 
publiek.wordpress.com and novini.nl. The source most circulated by regular 
users is the tendentious geenstijl.nl. 
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Figure4.8 Gephi visualization of PS2019 host-user network during the provincial 
elections campaign period, depicting only junk and tendentious hosts 
andthe users that circulate these sources 
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Figure 4.9 Gephi visualization of Party Leadership host-user network during 
the provincial elections campaign period, depicting only junk and 


tendentious hosts and the users that circulate these sources 
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In the issue space for the Utrecht shooting (Figure 4.7), tendentious and 
hyperpartisan sources (geenstijl.nl, tpo.nl and dagelijksestandaard.nl) 
populate the centre of the network. Several smaller clusters of junk news 
sources that have been circulated by regular users are evenly distributed 
on the periphery of the graph (e.g., drimble.nl (story-level), evendelen.net, 
dagelijksekrant.nl or hardwaarheid.nl). Only a minority of flagged users 
circulate clickbait (tpook.nl, nietbarkie.nl) and conspiracy pages (martin- 
vrijland.nl, ninefornews.nl, brekendnieuws.nl, ellaster.nl, wanttoknow.nl). 
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It is important to note that overall the hyperpartisan and tendentious 
sources in this network have been circulated by both flagged and regular 
users, making them appear to be mainstream(ing). 

The PS2019 (Provincial State elections) host-user network appears to 
be organized around two major hosts, hyperpartisan source opiniez. 
com and tendentious source geenstijl.nl (Figure 4.8). The (marginal) 
presence of clickbait host aboutmedia.nl is caused by the activity of only 
one regular user. Conspiracy hosts ninefornews.nl and dlmplus.nl have 
been only marginally circulated by users who also shared other junk 
news hosts. Two recently created user accounts in the network (created 
in December 2018) demonstrate an uncommonly high number of tweets 
and likes. One of them has around 39,300 posts, and 31,900 likes within 
four months of existence, a level of activity that suggests automation and 
artificial inflation. 

For the Party leadership network, the tendentious-hyperpartisan source 
tpo.nl and hyperpartisan source dagelijksestandaard.nl are the largest 
nodes in the network and are circulated by both flagged and regular users 
(Figure 4.9). Smaller nodes of hyperpartisan sources, such as fenixx.org, 
opiniez.com and verenoflood.nu, are positioned slightly more towards the 
periphery of the network. A dense cluster of flagged users is situated in the 
heart of the network and has circulated mostly tendentious and hyperpar- 
tisan hosts as well as conspiracy hosts, such as ninefornews.nl or ellaster.nl. 
Regular users populate the rest of the network and have circulated mostly 
tendentious and hyperpartisan hosts (e.g., tpo.nl, dagelijksestandaard.nl 
and opiniez.com) and to a lesser extent, have circulated conspiracy hosts 
(e.g., donquijotte.wordpress.com or stoppasfamiliedrama.blogspot.com) 
which are visible in the margins of the graph. 


Conclusions: Troll-like activity in divisive issue spaces 


As emphasized in studies of the campaigning by the Russian Internet 
Research Agency as well as so-called home-grown actors, Twitter allows 
for easy automation, which makes the platform susceptible to abuse by 
bot and troll-like users (boyd et al., 2018; DiResta et al., 2018; Howard et 
al., 2018). We have identified such suspicious activity during the Dutch 
Provincial elections of 2019, when looking at political issue spaces as well 


4 Their high number of likes is also inconsistent with the pattern of activity, which is mostly 
retweets and replies with GIFs or funny images. 
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as divisive issues. In fact, troll-like users are central across political and 
issue spaces around MH17, Zwarte Piet, Utrecht, PS2019, and the Dutch 
party leadership. In particular, 14 flagged users were found to be active 
across all political and issue spaces, and the 29 that appear in four out 
of five, deserve further scrutiny. Four suspect users active during the 
provincial election period were also (or still) active in all issue spaces 
during the EU election period. Some of these users had already been 
flagged in previous research from 2017, which means they have been 
operating and engaging in new and existing issues for over two years. 
Overall, our study found that such flagged users tend to spread mostly 
hyperpartisan and tendentious sources, followed by conspiracy websites. 
We also found no indication of a coordinated campaign, whereby (as found 
elsewhere) the troll-like users would include sock puppets, automated 
accounts, and semi-automated user accounts that post both retweets 
and original content. 

Divisive issue spaces are active year-round. From 18 February — the begin- 
ning of the official campaign — to 25 March 2019, the issue spaces of Zwarte 
Piet and MH17 were still active, even though Sinterklaas, the holiday related 
to Zwarte Piet, takes place in December and the downing of the Malaysian 
airliner was not in the news, either through new developments or official 
memorial events. A significant number of the most active users in each issue 
during this period display troll-like behaviour through their high activity 
(30% in the case of Utrecht and 60% in MH17 and Zwarte Piet). Despite the 
activity, most of these users’ influence is still limited, however. Only two 
of them appear among the top ten most @-mentioned for each issue space. 

At the same time, we identified at least three highly active new ac- 
counts that were created close to the elections with a clear purpose of 
disseminating divisive content, indicating how the platform may be 
employed around election time. When these troll or bot-like users are 
not aggressively attacking the opposition, they function as amplification 
machines for web news operations, ranging from tendentious sources 
such as Geenstijl and The Post Online to hyperpartisan sources such as 
De Dagelijkse Standaard, Opiniez and Fenixx. Repeatedly, we have seen 
how these tendentious and hyperpartisan sources are widely circulated 
by regular users who crowd out the flagged users (in a network cluster- 
ing sense). The uptake of tendentious and hyperpartisan sources by 
such regular users leads to a ‘mainstreaming’ of these hosts, in times 
of elections. 

In all, flagged users tend to spread mostly tendentious and hyperpartisan 
hosts, followed by conspiracy hosts, which appear in all datasets but seem to 
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be more pervasive in tragedy spaces as MH17 and the Utrecht tram shooting. 
During the EU election period, on several occasions, junk news sources 
outperformed mainstream sources around the controversial topics, Zwarte 
Piet and MH17. On both issues, junk news outperformed mainstream news 
in two of the five weeks. During these weeks, there is not a large increase 
visible in the engagement of junk news sources compared to other weeks. 
Instead, the overperformance is mostly caused by a drop in the mainstream 
media attention for the topics on hand, while coverage persists on the junk 
news sources, fuelling the debate. 

According to these results, the Dutch political Twittersphere does not 
appear to have a junk news problem, though it is populated by some troll- 
like users, whose existence serves to amplify certain voices. While we did 
not find a professional or large-scale trolling campaign, the activity across 
issues in spreading divisive content was caused by various types of user 
accounts, both bot-like (as in: automated) and troll-like (as in: repeatedly 
engaging with divisive issues and targeting politicians). Divisive issues 
remain steadily (even if marginally) active in junk news and tendentious 
news throughout the tested time frames, suggesting these issues are year- 
round rather than event-based or seasonal (as may be expected with Zwarte 
Piet). 


Appendix 4.1 Alternate figures 


Alternate Figure 4.2 These line graphs visualize the engagement with mainstream 
news (blue) and junk news sources (pink) during the Dutch 
provincial election campaign (PS) and the European Election 
campaign period (EU), similar to Figure 4.2, but excluding the 
tendentious-hyperpartisan sources. 
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Alternate Figure 4.3 These line graphs visualize the engagement with 
mainstream news (blue) and junk news sources (pink) for 
the issues of MH17 and Zwarte Piet during the provincial 
elections (PS), and the EU elections (EU), similar to Figure 4.3, 
but excluding the tendentious-hyperpartisan sources. 
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5 Dutch political Instagram 
Junk news, follower ecologies and artificial amplification 


Gabriele Colombo and Carlo De Gaetano’ 


Abstract 

The research examines junk news, followers of problematic sources 
as well as artificial amplification on Instagram during the 2019 Dutch 
provincial and European parliamentary elections. First, this study looks 
at the circulation of junk content in high-engagement political spaces on 
Instagram. Second, it takes up the question of the mainstreaming of Dutch 
junk news providers by looking at the intersection between the followers 
of Dutch political entities and those of junk news sources. Third, it looks at 
the presence of artificial engagement tactics (specifically fake followers) 
employed by Dutch political entities and news sources on Instagram. 
In all it was found that Dutch political Instagram is a relatively healthy 
space, but not for all issues or political entities. 


Keywords: Instagram, artificial engagement, junk sources, fake followers, 
digital methods 


Introduction: Fake followers, computational propaganda and 
their detection on Instagram 


Though Facebook has been labelled the ‘hyperpartisan media machine’ 
(Herrman, 2016) and Twitter studied as a matter of routine, owing to the 
availability of datasets, Instagram, when scrutinized, has been found 
to perform well as an outlet for junk or hyperpartisan news circulation, 
artificially amplified engagement and other types of problematic content 
and users. 


1 The research was undertaken together with Rama Adityadarma, Joris van Breugel and Vic Krens. 
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The relationship between Instagram and different types of ‘problematic 
information’ (Jack, 2017) has been studied in connection with the Russian 
effort to influence the 2016 American elections. A study by New Knowledge 
(2018) found that Russian propaganda tactics played well on Instagram. The 
report analyses data from a variety of platforms, in order to detect efforts by 
the Internet Research Agency (IRA) to spread disinformation and divisive 
content. The study found that Instagram, with ‘187 million engagements’ of 
‘116,000 Instagram posts across 133 accounts’ (New Knowledge, 2018: 7), to be 
at the forefront of an IRA operation, with better performing fake accounts 
and overall higher engagement than on Facebook. 

The significance of Instagram, which ‘outperformed Facebook’ (New 
Knowledge, 2018: 8) as a battleground in the Russian disinformation enter- 
prise, is linked, according to the report, to two possible causes. First, since 
it is a platform designed around sharing visual materials, Instagram may 
be well suited for the so-called ‘image-centric memetic (meme) warfare’ 
(2018: 8), that is, the weaponized use of image macros to stir conflict and 
foster division online. Second, the report states that the considerably high 
engagement of content from the IRA’s accounts on Instagram may also be 
the result of click-farm activity, and some of the accounts in the dataset 
appeared indeed to be linked to ‘a live engagement farm’ (2018: 8). 

With respect to the Dutch case, Russian influence has been studied 
mainly on Twitter, with the detection of trolling activities, especially in 
the aftermath of tragic and divisive events both in the Netherlands and 
in Belgium. For example, two journalistic studies found peaks in Russian 
trolling activity following the downing of MH17 in 2014 (Kist and Wassens, 
2018; van der Noordaa and van de Ven, 2018a), while another study uncovered 
a (rather unsuccessful) organized Russian effort in spreading anti-Islam 
content on Twitter after the 2016 Brussels airport attacks (van der Noordaa 
and van de Ven, 2018b). Despite the lack of empirical research regarding 
Russian influence on Instagram, one study from the NRC Handelsblad (Kist 
and Wassens, 2018) suggests that a larger organized trolling activity may be 
found on other platforms beyond Twitter, including Instagram. 

The use of computational means to amplify misinformation and hy- 
perpartisan content on Instagram has not been linked exclusively to the 
Russian propaganda operation in the West, but it has also been described 
as a domestic tactic, adopted by national campaigners as well. A compara- 
tive, global study of social media manipulation in 48 countries (Bradshaw 
and Howard, 2018) describes different computational tactics for political 
influence online, including the use of fake accounts to attack other users, 
automated accounts generating artificial engagement, and human-curated 
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accounts that employ automation to be more efficient. With respect to the 
Netherlands, the study found such automated activity to be mainly linked to 
the boosting of Geert Wilders’ hashtags on Twitter. While the study describes 
Twitter as the platform where automation thrives the most, signs of ‘cyber 
troop activity’ (Bradshaw and Howard, 2018: 13) are also to be found in 
other platforms, including Instagram, among 25% of the countries studied. 

That fake following and artificial engagement flourish on Instagram may 
also be noted from reported cases in the news. As a case in point, in June 2017, 
the Russian journalist, Vasily Sonkin, posted an image of a vending machine, 
placed inside a shopping centre in Moscow, that lets users buy Instagram 
followers and likes. The news that for the (cheap) price of 50 Russian roubles 
(about EUR 0.70) one could buy 100 fake Instagram likes was reported by 
numerous tech or news media outlets (Matsakis, 2017; Feldman, 2017; Tan, 2017). 

There have also been efforts by Instagram itself to counter artificially 
amplified activities on the platform. In December 2014, Instagram an- 
nounced a crackdown on fake (or improperly obtained) profiles, in the 
so-called ‘Instagram rapture’ (Lorenz, 2014) that resulted in the deletion of 
hundreds of thousands of accounts. And later, in December 2018, a ‘Christmas 
crackdown’ (Lorenz, 2018) resulted in the shutdown of 500 meme accounts, 
some of which with millions of followers, suspected of using stolen or traded 
profiles. On the same note, in April 2019, Facebook filed a lawsuit against 
one company based in New Zealand, accusing it of providing ‘fake likes, 
views and followers to Instagram users’ (Romero, 2019). The lawsuit is 
presented as part of a larger effort by the platform to prevent ‘inauthentic 
behaviour’ on Instagram. While actions have been taken to cope with 
artificial engagement as well as fake or improperly obtained accounts, the 
platform has been apparently less active in limiting the spread of extremely 
coloured or hyperpartisan content, but rather has become the ‘Alt-Right’s 
new favourite haven’ (Sommer, 2018), offering refuge to extreme right-wing 
personalities, after their accounts are deleted from Twitter. 

The presence of a large automated engagement infrastructure on Ins- 
tagram is also indicated by the deluge of fakeness-detection tools offered 
by commercial services online. The detection of inauthentic automated 
activity on Instagram may work by fingerprinting one account’s follower 
base by nationality, and associate specific geographical locations, such as 
Brazil, Turkey or China, to suspected bot activity (Maheshwari, 2018). For 
example, among the available tools, HypeAuditor, a 100% Al-powered’ 
service to expose ‘fake followers and engagement’ on Instagram, flags 
certain countries, such as Brazil, as geographical locations that may signal 
the presence of fake followers (Komok, 2018). 
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Despite being understudied, specifically in the Dutch context, Instagram 
appears to be a platform prone to the presence of various instantiations of 
junk and fakeness. There is the presence of content that can be described as 
false as well as merely hyperpartisan and divisive, but deliberately pushed 
online in order to stir conflict in a political space, both from outside the 
country and from within. It is also a platform prone to various computational 
tactics (such as bot work, fake likes and fake following) employed as a means 
to artificially amplify that same content. 


Junk content and artificial amplification in the Dutch political 
space on Instagram 


As mentioned, a few studies have described Instagram as fertile ground for 
the distribution of inflammatory content in the form of memes, but also as a 
well-performing infrastructure for the artificial amplification of engagement. In 
this empirical research project, we devised three complementary approaches for 
the assessment of Dutch political Instagram in order to test these premises. They 
study the amount of junk content shared on the platform, the dubiousness of 
the most relevant information sources within the space, and the inauthenticity 
of followers that may generate artificial engagement (see Figure 5.1). 

In the first part of this study, we search for levels of junk content shared on 
the platform, by asking to what extent the most liked content in a demarcated 
Dutch political space on Instagram can be defined as junk (i.e., disinforma- 
tion, conspiracy, clickbait or hyperpartisan). Second, as the estimation of junk 
also can be made through ‘online source criticism’ (Rogers and Niederer, this 
volume), we expand the work by detecting it on Instagram at a source level. 
Here, we study the mainstreaming of junk sources by exploring the affinity of 
the follower bases of Dutch political entities with those of junk news providers 
(flagged as such by experts). We ask, to what extent do Dutch political entities 
share an audience with junk news sources on Instagram? Thirdly, in order 
to study the tactics of artificial engagement that political parties and news 
sources may employ on Instagram to boost their content, we search for signs 
of inauthentic activity in the follower bases in the Dutch political space and 
inquire into the extent of any efforts at artificially boosting (by means of fake 
followers) present around divisive topics on the Dutch Instagram. 

In this research we employ a ‘digital methods’ approach (Rogers, 2013), 
repurposing Instagram-specific features to gather data from the platform. In 
particular, we compile a list of hashtags and profiles in order to demarcate 
the Dutch political space on Instagram. Within this space, we collect and 
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analyze most liked posts (i.e., posts that receive a certain number of likes) 
to study junk in the shared content. Furthermore, we collect followers of 
the accounts of Dutch political entities, mainstream news sources, and junk 
news sources, in order to study the intersection between their audiences, 
and more generally to assess the degree of junk in the Dutch political space 
at the level of the sources. Finally, we rely on profile features (such as the 
150 characters bio in one’s profile, or post captions), to evaluate the number 
of fake followers of the Dutch political space. 


Detecting junk in the most liked content 


The aim of the following analysis is to identify engaging content in the 
Dutch Instagram political space and observe the extent to which it contains 
junk — in the sense of how much engagement is generated by content that 
is either disinformation, conspiracy, clickbait or hyperpartisan. 

To outline the Dutch political space in Instagram, we compiled a list of 
hashtags (see Table 5.1) that are used on the social network to identify the 
leaders of Dutch political parties (e.g., #markrutte), the 2019 Dutch provincial 
elections (#PS2019) as well as politically charged issues such as climate change 
(#klimaatverandering). We used the Instagram Scraper tool,” offered by the 
Digital Methods Initiative, to collect the 1,000 most recent posts per hashtag 
(data collected between the 25th and 28th of March 2019), together with their 
metadata (date of the post, media URL, caption, number of comments and 
number of likes). For each hashtag we selected only the 20 most liked posts, 
manually filtering out posts that are not relevant to the search criteria, or 
identical posts that prevent more diverse results from reaching the top 20.3 

In this curated list of most liked posts, we conducted a close reading 
by looking at post captions and embedded media (images and videos) to 
understand how political party leaders and politically charged topics are 
discussed within the limits of the Instagram Dutch political space, and 
specifically to flag the presence of junk content. 

As a result of this evaluation (see Figure 5.2), we found that out of the 
400 most liked posts within our dataset there are (only) 45 posts that can 


2 The tool is available at this link: https://wiki.digitalmethods.net/Dmi/ToolInstagramScraper 
3 For example, we filter out posts about the Slovak professional footballer Marek Hamšík, 
who plays with the number 17 and is referred to in Instagram with the same hashtag of the 
Malaysia Airlines Flight 17 (#MH17). We also do not include in the dataset the identical posts 
of condolence messages for the Utrecht attack posted by Dutch national football team players 
with the hashtag #Utrecht. 
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be flagged as junk, 4 satirical posts, and 351 posts that do not appear to be 
junk. Looking at the engagement generated by these posts, junk content 
was liked 7o,466 times, satirical content 37,532 times, and non-junk content 
838,794 times. 

In Figure 5.3, the 4oo most liked posts are divided in hashtag-dedicated 
columns, in which they are also ranked from the most liked post in the 
first row to the least liked one in the last. Junk content is flagged using 
three different colours: light blue for hyperpartisan content, magenta 
for conspiracy, and blue for click-bait. Satirical posts are color-coded in 
dark blue. Finally, columns are ordered from left to right according to 
the amount of junk content, calculated on the total number of likes for 
each hashtag. 

The analysis shows that the #zwartepiet, #geertwilders and #tuna- 
hankuzu hashtags represent the most divisive political spaces, with 
respectively 56.1%, 42.8% and 42.7% of their total amount of likes directed 
to junk content. Moreover, we find that the majority of the posts flagged as 
such can be considered hyperpartisan, mostly supporting and/or oppos- 
ing particular ideology or figures, while only one post can be considered 
as clickbait, and one conspiracy. Generally, we did not find any trace of 
disinformation linked to the content that receives the most likes. The 
findings suggest that certain issues or political leaders, such as the Zwarte 
Piet debate and the leader of Denk political party, Tunahan Kuzu, draw 
more divisive content than others. Of the 20 most liked posts, however, we 
found no strong presence of junk. 

In general, we found a relative scarcity of junk content in this high- 
engagement political space. In the top results for the Dutch provincial 
elections, #PS2019, we found only positive content, either celebrating 
preliminary poll results or encouraging people to exercise their right to vote. 
The hashtags, #24oktoberplein and #utrecht, returned mainly condolence 
posts and the news that the attacker was spotted and arrested. Almost 
all of the content we considered as junk is hyperpartisan. We found no 
presence of disinformation in the most liked results within the demarcated 
political space. 

In order to ascertain the presence of junk content on Instagram sur- 
rounding the 2019 European Parliamentary elections in the Netherlands, 
we conducted a second hashtag analysis concerning content posted in the 
months before the election day (23 May). With the goal of demarcating the 
Dutch political space around the 2019 European elections, we compiled a 
new list of hashtags (see Table 5.2) used to identify Dutch political parties 
(e.g. #fvd) and their leaders (e.g. #thierrybaudet), the European elections 
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Table 5.1 Lists of hashtags pertaining to political leaders and politically charged 
discussions used to demarcate the Dutch political space on Instagram 


around the 2019 provincial elections 


Hashtags related to 
Dutch political party leaders 


Hashtags related to 
politically charged discussions 


#markrutte, #rutte, #geertwilders, 

#wilders, #thierrybaudet, #baudet, #jetten, 
#tunahankuzu, #jesseklaver, #lodewijkasscher, 
#alexanderpechtold, #gertjansegers, #sybrand- 
buma, #mariannethieme 


#PS2019, #klimaatverandering, 
#immigranten, #utrecht, #zwartepiet, 
#MH17, #24oktoberplein 


Figure 5.2 Proportions of most liked content shared around the 2019 Dutch 


provincial elections, categorized as junk, satire, and not junk 


Top 20 most liked posts per hashtag (400 posts in total) 


Not Junk B Junk B Satire 
11,25% 831% 
3,93% 
37.532 likes 
400 posts 955, 792 likes 
87,75% 87,76% 
361 posts 838704 Ror 


Amount of junk 
and satirical posts 


Amount of likes 
of junk and satirical posts 


Data source: Instagram Scraper; data collection: 25-28 March 2019; pie charts 


(#EUverkiezingen2019, #EUverkiezingen), and various politically charged 
issues such as immigration (fimmigratie, #migratie, #immigranten) and 
climate change (#klimaat, #klimaatverandering). 

With the Instagram Scraper tool, we collected the 1,000 most recent 
posts per hashtag (data collected on the 22nd of May) and their metadata. 
For each hashtag we only retained posts shared after the 28th of March, 
in order to focus on the detection of junk in the period prior to the Euro- 
pean elections, but after that of the Dutch provincial elections. For each 
hashtag we selected the 20 most-liked posts, excluding those included in 


(most junk) to left (least junk) 


Af 


Figure5.3 20 most-liked posts per hashtag shared around the 2019 Dutch provincial elections, sorted from right 
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Figure 5.4 Examples of the posts flagged as hyperpartisan or satire 


HOW TO RUIN À 
FIRST DATE 
IN 3 WORDS 


Data source: Instagram Scraper; data collection: 25-28 March; image wall 


the dataset but not relevant to the search criteria.* Subsequently, in order 
to ascertain the amount of junk in the dataset, we looked at embedded 
media and textual captions and flagged each post as junk or not (making 
the additional distinctions between disinformation, conspiracy, clickbait 
and hyperpartisan content). 

The analysis (see Figure 5.5) confirmed the relative lack of junk content 
in the Dutch political space, also around the 2019 European parliamentary 
elections: out of 452 most liked posts, we found only 41 that can be considered 
junk (specifically hyperpartisan), counting for less than 10% of the total 
amount of posts. Moreover, hyperpartisan posts score low even in terms of 
engagement, generating only 4.66% of likes out of the total amount. 

In Figure 5.6, most liked posts are organized in hashtag-dedicated col- 
umns. Columns are grouped by type of hashtag and sorted from right to left 
according to the number of likes generated by hyperpartisan content. The 


4 The query for some of the less popular hashtags returned less than 20 posts in the specified 
date range. 
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Table 5.2 Lists of hashtags pertaining to political leaders and politically charged 
discussions used to demarcate the Dutch political space on Instagram 


during the months before the 2019 European elections 


Hashtags related to Hashtags related to Hashtags related to 

Dutch political party Dutch political parties politically charged 
leaders discussions 

#markrutte, #rutte, #cdavandaag, #pvv, #social- #duurzaamheid, #klimaat, 
#geertwilders, #wilders, istischepartij, #pvda, #chris- #klimaatverandering, 
#thierrybaudet, #baudet, tenunie, #partijvoordedieren, #immigratie, #migratie, 
#jesseklaver, #jetten, #50pluspartij, #groenlinks, #immigranten, #mh17, 
#mariannethieme, #fvd, #stempiraat, #voltneder- #zwartepiet 
#tunahankuzu land, #d66, #degroenen 


Figure 5.5 Proportions of most-liked content shared around the 2019 European 


elections, categorized as junk and not junk 


Top 20 most liked posts per hashtag (452 posts in total) 


Not junk mg Junk 
9,98% 4,66% 
41 posts 10.208 kes 
452 posts 433,430 likes 
90,02% 95,34% 
411 posts 414.122 thes 
Amount of junk posts Amount of likes 


of junk posts 


Data source: Instagram Scraper; data collection: 22 May 2019; pie charts 


analysis shows that the hashtags related to political parties attracting more 
divisive content are #pvda and #christenunie, with respectively 27.6% and 
25.1% of likes directed to hyperpartisan content. Compared to the dataset 
around the provincial elections, Geert Wilders (#geertwilders, #wilders) 
remains the political leader receiving the highest percentage of likes directed 
to hyperpartisan content (28.4%), followed by Jesse Klaver (#jesseklever) 
with 13.4%, who instead scored low in terms of junk content in the previous 
analysis. We did not find traces of hyperpartisan content in the most-liked 
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posts around other political leaders. As was the case with the hashtags 
used to refer to the Dutch provincial elections, #EUverkiezingen2019 and 
#EUverkiezingen are related mainly to invitations to exercise the right to 
vote. Among the issues under study, #zwartepiet remains the most divisive 
one, with 22.9% of likes directed to hyperpartisan content. 

In general, we did not find evident signs of dubiousness in the most-liked 
content around the 2019 European elections, except for a few hyperpartisan 
posts. The finding is aligned with that of the hashtag analysis conducted 
around the 2019 Dutch provincial elections. 


Follower ecologies and the relevance of junk sources 


In order to detect the relevance of junk news sources within the Dutch 
political space on Instagram, and to assess whether and how much junk news 
sources are becoming mainstream, we studied the overlap between followers 
of Dutch political entities, mainstream news sites and Dutch-language junk 
news sites.5 Specifically, we asked, to what extent are followers of junk news 
providers shared with those of Dutch political entities? 

First, we demarcated the Dutch political space on Instagram, by compiling 
three lists of profiles: a list of Dutch political parties and their leaders, a list 
of Dutch mainstream media outlets, and the profiles of Dutch information 
sources flagged as junk in the expert list (see Appendix 6.2 in Hagen and 
Jokubauskaite, this volume). We then used the API Instagram Follower 
Collector by Phantombuster’ to collect the follower list of each Instagram 
account, and then, by creating a co-follower network, we looked at the 
amounts of shared followers between the political entities and the dubious 
Dutch information sources from the expert list. 

In mapping the follower network of the Dutch political space, we found 
three distinct follower ecologies (see Figure 5.7). First, an ecosystem of 
followers of mostly established mainstream news organizations, such as 
the Dutch public broadcasting station, NOS. The follower bases of these 
news organizations are the largest in the network, which suggests that the 
Dutch mainstream news providers are still more relevant that those flagged 


5  Weuse thelist of sites flagged by the Hoax-Wijzer (www.hoax-wijzer.be), which was edited 
and enhanced by University of Amsterdam researchers, and is dubbed the ‘expert list’ (see 
Appendix 6.2 in Hagen and Jokubauskaite, this volume). 

6 Phantombuster is an API store that ‘provides ready-made cloud APIs to collect data from 
various social networks and improve marketing strategies’ (phantombuster.com). 
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as junk, at least in terms of follower count. Few sites from the expert list 
are close to (or part of) the cluster of mainstream news organizations, due 
to a relatively high number of shared followers. Shared followers among 
mainstream news organizations and junk news sites may indeed suggest a 
special affinity among them, or rather be the signal of the mainstreaming 
of junk news providers. 

A second ecosystem is made up of political parties and their youth organi- 
zations. The distribution of parties is laid out from left-wing to right-wing 
parties, whilst still being tightly clustered together. This may suggest that 
most followers either follow multiple parties on the same side of the political 
spectrum or follow all political parties regardless of political leaning. What 
can also be observed is the relative distance of the cluster of political parties 
to that of news organizations, suggesting that followers of political entities 
are mostly not shared with those of news organizations. 

A third cluster is made up of right-wing political entities, which are 
far from other political entities, closer to few hyperpartisan or clickbait 
sites and to few, less established, mainstream news providers. Within this 
cluster, the account of PVV leader Geert Wilders is surrounded by GeenStijl, 
a tendentious ‘shock blog’ and PowNed, the public broadcasting station that 
is an offshoot of GeenStijl. The official profile of FvD (Forum for Democracy) 
and the youth organization of the same party are even more distant and 
isolated from other parties: they are surrounded by individual political 
commentators and share a high number of followers with the hyperpartisan 
news site, De Dagelijkse Standaard. This topology may suggest that although 
these parties and personalities share some followers with those from other 
sides of the political spectrum, they are mostly on their own and produce 
content consumed by a unique audience. 


Fake followers and artificial engagement 


In order to profile the follower base of the previously demarcated Dutch 
political space, we feed each account? (of political entities, but also of 
mainstream media, and of those from the expert list) in the HypeAuditor 
tool to check the authenticity of the accounts and look for signs of artificial 
boosting and fake followers. With HypeAuditor one can profile an Instagram 
account to determine the authenticity of its follower base. To assess the 


7 HypeAuditor analyzes only accounts with more than 1,000 followers. For this reason, we 
limited the detection of fakeness to accounts with more than 1,000 followers. 
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Figure 5.7 Follower ecologies in the Dutch political space, visualized as a co-follower 
network and manually annotated. In the network, accounts with higher 


amounts of shared followers (pink) are placed closer to each other. 


(© Mainstream media 
e @ Political entities 

Size represents 

the number 

of followers Followers 


@ Junk sources 


trendnieuwstv 


MAINSTREAM MEDIA 


nieuwetij@§kindmagazine 


vrouwendingenonline 


telegraaf.nl nrenl 
ad_nl nos de_volkskrant 


' 
' 
' 
' 
' 
i 
! 
' 
trouw.deverdieping 


' 
healtywatch_nt ' 
| 


` © _ 


IW" "Cen ee "e, am Zem ET am t T pateren e N 
! destilewaarheid pink. ee 

l ea leeshetnu i 

' Wi ' 

' ia š ' 

! Lái gire! rnew: fs, 1 

trendnova rl nl de p Ge E partikiars 

! p! stevebrawnamsterdamnoir ` e " roodiongindesp { 

Ó Ch partijvaidearbeld i 

` my t 

Y omroeppowned 3 we ositie; t 

likemag " ' Ome Jada 

$ thepostonlirle daa i 

S geenstijl SS on jongégemocraten | 

' 

` rs i 

` S pfiieuws H 

` e° g cujøngeren H 

Ç ' 

x sgpjongeren i 

E dedagelilsestandaard I 

' 

' 

' 


POLITICAL PARTIES 


ALTERNATIVE MEDIA ECOSYSTEM 


Data source: Phantombuster; data collection: 25-28 March; network graph 


extent to which Dutch political accounts are employing artificialengagement 
tactics, we use reports from HypeAuditor, regarding the percentage of real 
followers, and their geographical origin.® The percentage of fake followers 


8 According to HypeAuditor, the geographical origin of one follower base is detected by 
analysing profiles biographies and place names in post captions (twitter.com/hypeauditor/ 
status/1077143110432538624). 
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Figure 5.9 Visualization of the follower base of Mark Rutte's personal and work 
accounts and Geert Wilders’ account, based on results from the 
HypeAuditor tool. Each follower base is segmented based on ‘audience 
type'and geographical provenance. Popular suspicious countries, that 
may suggest an inauthentic follower base, are coloured in red. 


Audience type 
Real People B influencers B Mass Followers MB Suspicious Accounts 
Mark Rutte Mark Rutte (Personal) Geert Wilders 
40.7K followers 49.7K followers 73K followers 
13.8% 


22.8% 26.7% 
2.3% 
07x 0.9% 


Followers main countries 


Netherlands 52% Netherlands 56% irar 36% 
irar ax United States 4% Netheriardis 26% 
Vere ruola Ki Germary 3% Turkey aa 
naty 2% France 3% unned States xx 
France 2x United Kingdom 2% Germany 2% 


+ popular locabons for cack and fofiowers farms 


Data source: HypeAuditor; data collection: 25-28 March 2019; pie charts 


returned by the tool is then used to rank each account from less fake to 
more fake (see Figure 5.8). Furthermore, we zoomed in on those accounts 
with a higher percentage of fake followers, to observe their geographical 
provenance (paying particular attention to suspicious countries), as well 
as the segmentation of the follower base provided by HypeAuditor, which 
breaks down followers in ‘real people’, ‘influencers’, ‘mass followers’ and 
‘suspicious accounts’ (see Figure 5.9). 

Generally, we found that the majority of profiles do not have a suspicious 
follower base, with most accounts scoring higher than 70% in the real 
follower metrics provided by the tool. There are some accounts, however, 
that are suspect of having a fake follower base. For instance, the media 
entity PowNed has 32.6% of suspicious followers. The clickbait site Prankster 
also scores relatively high in terms of fake following. Within the group of 
political entities, the personal account of Mark Rutte and the account of 
Geert Wilders have the highest number of suspicious followers. Strikingly, 
the ‘work’ account of the prime minister, Mark Rutte, has a lower percentage 
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of fake followers than that of his personal account. On the other hand, the 
account for the political party, Christenunie, has hardly any suspicious 
followers, just as the SGP (Reformed Political Party) and that of the minister 
Gert-Jan Segers. 

When we look closer to the nationality of the follower bases, we found 
no suspicious results, with most of the accounts followed by users based 
in The Netherlands. For both of Mark Rutte’s accounts, the followers are 
mostly based in the Netherlands. On the contrary, Geert Wilders account 
has 36% of his followers from Brazil. This raises some questions regarding 
the legitimacy of Geert Wilders’ follower base, for Brazil is often mentioned 
as one location that can signal the presence of fake followers (Maheshwari, 
2018). 

In all the follower analysis does not show an organized effort of artificial 
boosting within the Dutch political Instagram sphere, and it indicates, with 
the exception of Geert Wilders, a rather authentic follower base. 


Conclusions: Findings and limitations 


The goal of the present research is to detect the scope of junk news and the 
degree of artificial amplification in the Dutch political Instagram sphere. 
More generally, it can be considered an attempt at applying to the Dutch 
context the argument in the New Knowledge report (2018) that Instagram 
performs well in terms of junk content circulation and artificial amplifica- 
tion strategies. It also takes up the invitation from the NRC Handelsblad study 
to inquire into other platforms than Facebook and Twitter for disinformation 
campaigning and computational propaganda. 

The presence of dubious content (or lack of thereof) has been studied on 
three levels: at the story level (by looking at the circulation of junk content 
in high-engagement political spaces on Instagram); at the source level (by 
looking at the intersection between the follower bases of Dutch political 
entities and that of news sources flagged as junk); and through the detection 
of artificial engagement tactics, specifically fake followers, among the 
profiles of Dutch political entities as well as Dutch information sources. 

In general, we found a rather healthy political space. Most liked content in 
the Dutch political space proved to be junk to a very small degree, although 
we found a small amount of hyperpartisan and polarizing content centred 
around more divisive figures and issues in the 2019 Dutch provincial and 
European elections. With respect to the alignment of the audience of Dutch 
political parties with that of (mainstream or junk) news providers, we found 
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mainstream news organizations to be still more relevant in this political 
space,somehow confirming the argument thatin the Netherlands “the vast 
majority of news consumption remains of the mainstream sources' (Rogers 
and Niederer 2019, this volume). Furthermore, the analysis of the follower 
base of Dutch political entities (and that of news sites, both mainstream 
and junk) revealed an apparent authentic audience with almost no signs 
of artificial engagement. 

Within a relatively healthy political spectrum, it is at the extremes 
that junk sources and artificial amplification surface. With the cur- 
rent research we have pointed out a special affinity between right-wing 
political entities and some information sources that may be defined as 
junk (or at least hyperpartisan). Furthermore, the few indications of 
artificial engagement we have found are located at the far end of the 
political spectrum, with Geert Wilders’ account being the most suspected 
of inauthentic activity. 

In the co-follower analysis, we found that extreme political entities to 
have a unique follower base, not shared with other parties or mainstream 
news sites. Right-wing political entities are also relatively closer (in terms 
of shared followers) to suspicious sources (a few of them flagged by the 
expert list). Above all, Geert Wilders’ account is the closest (according to 
shared followers) to hyperpartisan news sources. Relatedly, Geert Wilders’ 
account is the only one of those under study that may reveal signs of artificial 
engagement, as suggested by a geographically dubious follower base. This 
finding resonates with the 2015 scandal about a suspicious increase of the 
follower count of Geert Wilders’ Twitter profile. In addition, the already 
mentioned comparative study of social media manipulation strategies by 
the Oxford Internet Institute (Bradshaw and Howard, 2018) also refers to 
Geert Wilders as making use of various artificial boosting strategies in 
The Netherlands, reporting on an analysis by a social media analytics firm 
that in February 2016 found 26 fake accounts amplifying the #geertwilders 
hashtag on Twitter. 

The determination of the relative absence of junk content, dubious 
sources and fake followers in the scope of the current research has a series 
of methodological limitations. First, in the search for junk news in the shared 
contents, we collected data based on a limited list of hashtags related to 
Dutch politicians and controversial topics. One could repeat the analysis to 
include other politically charged issues. Furthermore, we have considered 
only the top 20 most-liked posts per hashtag, whereas we could have also 
counted the number of comments per posts to analyze most engaged-with 
content. Moreover, we could have included in the analysis a larger set of 
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posts that do not necessarily make it to the top (because they receive fewer 
likes, or have fewer comments), in order to evaluate the presence of junk 
newsin less engaged-with spaces. In addition, given that for data collection 
we made use of the DMI Instagram Scraper, which “scrapes Instagram to 
retrieve posts’ (Digital Methods Initiative, 2019), this research is dependent 
on the limits of such scraping, including Instagram’s rate limits which are 
not documented and unknown security challenges (Instaloader, 2019). It is 
also not a platform that invites research through scraping. As others have 
pointed out, social media platforms are designed to increase a platform’s 
commercial value, rather than to meet researchers’ needs (Borra and Rieder, 
2014). To overcome the limitations, one could use additional tools for data 
collection and compile a richer data set. 

Secondly, we established the fakeness in the Dutch political follower base 
using the metrics provided by a single tool (HypeAuditor). We could have 
compared the results with those by other similar services (and audited the 
auditors, so to speak). Moreover, we searched for signs of inflated engagement 
in the Dutch political space only by looking at followers’ demographics, while 
we could have paid attention to other signals such as patterns of repetition 
in posts comments. For example, to account for other tactics of artificial 
engagement on Instagram, one could perform a co-hashtag analysis? in a 
demarcated issue space, and detect signs of (semi-automatic) boosting, such 
as the use of long list of popular unrelated hashtags, deliberately added in 
the post captions to increase content visibility.’° Moreover, one could trace 
back the users involved in this activity and profile them in order to evaluate 
their authenticity. 


9 _ In addition to the most recent lists of posts, the Instagram Scraper tool returns a network of 
hashtag co-occurrences, that is, a file that contains the hashtags used at least once together with 
the hashtag under study. For each pair of hashtags, the tool returns a numeric value representing 
the total number of posts in which the two hashtags appear together in the data set. A similar 
approach is largely used for empirical research on Twitter: with co-hashtag analysis one can gain 
a sense of the relationship between subtopics in a conversation (Borra and Rieder, 2014); or find 
additional and/or more ‘significant hashtags’ (Rogers, 2017) to be queried to expand a corpus of 
data; or spot hashtags practices aimed at enhancing the visibility of particular content (Wang et 
al. 2016), or overturning its original meaning through hashtag hijacking practices (Berg, 2017). 
10 Unlike Twitter, which has a character limit of 280 characters, Instagram’s character limit is 
2,200 characters, and users can include up to 30 hashtags in the caption and comment sections 
of the post. This results in certain users adding blocks of more or less related hashtags to the 
posts to enhance their visibility. Even if Instagram is applying countermeasures to block the 
use of certain hashtags (Drewe, 2016), there are several websites that provide lists of safe and 
popular hashtags that users can copy paste directly in their posts (for example, tagblender.net). 
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6 Dutch junk news on Reddit and 
4chan/pol 
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Sal Hagen and Emilija Jokubauskaité 


Abstract 

This chapter investigates the presence of junk news on Reddit and 4chan’s 
/pol/ subforum, spaces often described as “alternative” owing to their lower 
user numbers and subcultural ethos compared to the likes of Facebook. 
We first delineate Dutch spheres within the two spaces over multiple 
years, finding a rising number of posts within Reddit’s Dutch sphere and 
a stagnant yet non-negligible number of Dutch posters on 4chan/pol/. We 
then categorise and analyse what URLs are shared to gauge the presence 
of junk news domains. We find that Reddit seems fairly resilient against 
the presence of disinformation or other forms of junk news, save for the 
appearance of some hyperpartisan sources and incidental malicious users. 
4chan/pol/ shows a somewhat more problematic situation, returning a 
larger presence of (foreign) junk news sources. 


Keywords: Reddit, 4chan/pol/, junk news, alternative media, digital 
methods 


Introduction: The understudied, deep vernacular Web 


Recent debates on online fake news and disinformation have largely been 
discussed with respect to the social media behemoths in the context of a 
‘platformized’ internet ecosystem (Helmond, 2015), with Facebook, Twitter, 
Instagram and YouTube in the spotlight. It is not without reason; given their 
gigantic user bases, open publishing and micro-targeting, they are vulnerable 


1 The research team includes Lucie Chateau, Gabriele Colombo, Ognjan Denkovski, Carmen 
Ferri and Holly Foxton. 


Rogers, Richard, and Sabine Niederer (eds), The Politics of Social Media Manipulation. Amsterdam, 
Amsterdam University Press 2020 
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to disinformation campaigns and dubious information, not so unlike the 
Web itself. Regardless, they do not exist in a vacuum. On the fringes of the 
Web, yet difficult to characterize as marginal, are pseudonymous or anony- 
mous platforms like Reddit and 4chan. Instead of public-facing ‘e-celebs’ or 
otherwise identifiable accounts, these spaces are characterized by ‘masked’ 
users with distinctive subcultural styles, vernaculars and iconographies. 
The pseudonymous and anonymous users on Reddit and 4chan do not only 
congregate around shared interests or common goals, but also — and in some 
cases predominantly — around a deep understanding of shared subcultural 
knowledge and norms. The unconventional and sometimes downright eso- 
teric cultural productions some of these groups create feed into community 
members’ self-imagination as ‘underground’, ‘countercultural’, or ‘internet 
native’. 4chan and (parts of) Reddit can be associated with the term ‘deep 
vernacular Web’ (Tuters and De Zeeuw, 2019), referring to online discussion 
forums that lack stable user identities and whose masked participants 
frequently transgress the boundaries of ‘mainstream’ conventions, often 
through an entangled mix of sincere ideology and ironic play. 

While Facebook, Twitter, Instagram and YouTube have already been studied 
in relation to issues of ‘fake news’, the abovementioned ‘virality-oriented 
subcultures’ of the deep vernacular Web are also said to play a ‘crucial role in 
the system’ of the circulation of various types of ‘junk news’ (Venturini, 2019). 
4chan and certain parts of Reddit have indeed been characterized as hotbeds for 
disinformation (Shiebel, 2017; Collins and Russell, 2018; Lagorio-Shafkin, 2018), 
trolling campaigns (Phillips, 2015), and conspiracy theories (Marwick and Lewis, 
2017; Tuters et al., 2018). Despite their relatively marginal number compared to 
more mainstream platforms, users of Achan and areas of Reddit are considered 
particularly skilled in ‘setting the agenda’ of broader news media (Phillips, 2018). 
In a 2017 report, Marwick and Lewis highlight how an underground current 
of Internet subcultures associated with 4chan and Reddit ‘take advantage of 
the current media ecosystem to manipulate news frames, set agendas, and 
propagate ideas’ (Phillips, 2018: 1). Later, Phillips builds on this research by 
exploring how and why the false narratives of these online antagonists were 
amplified by major U.S. news outlets (2018). As she identifies, journalists were 
keen on reporting the narratives with false information or dark undertones 
partly because ofa fascination with their bizarre cultural phenomena or simply 
due to a lack of time required to decipher their problematic code language. 
The reporting, she argues, ‘amplifies’ their overall presence. By 2019, there 
are now well-known by-products of this cycle of the normalization of false 
content emerging from fringe online spaces. To provide but one example, the 
‘Pizzagate conspiracy theory, originating on 4chan, presumed the Clintons 
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were maintaining a child sex trafficking ring, which led to media coverage and 
anactualshooting in a US pizza parlour (Tokmetzis, 2o18; Tuters et al., 2o18). A 
related conspiracy theory, ‘QAnon’, gradually spread from 4chan to Reddit and 
mainstream news sources (Hagen et al., 2019), even sprouting international 
support groups including ‘QAnon Nederland’ (QAnon Netherlands). 

The influence of fringe internet subcultures on the news ecosystem has 
mostly been scrutinized in relation to English-language spheres and U.S. 
politics. As such, it remains unclear to what extent the propagation of prob- 
lematic content from the ‘deep vernacular Web’ affects other news ecosystems 
such as the Dutch. Such activity already has caught the attention of Dutch 
media outlets. For example, the QAnon conspiracy was covered by major 
outlets like RTL Nieuws (2018) and Algemeen Dagblad (Van Huet, 2018), while 
De Correspondent untangled the related Pizzagate conspiracy in some depth 
(Tokmetzis, 2018). De Volkskrant discussed Dutch users active in the far-right 
‘politically incorrect’ subforum of 4chan, /pol/, by observing an increasing 
prevalence of anti-Semitic conspiracies (Kranenberg and Bahara, 2018). Ina 
broader sense, conspiratorial rhetoric native to the deep vernacular Web seems 
to be normalizing in the Dutch political and media discourse at large. For 
instance, the concept of ‘cultural Marxism’ has increasingly appeared in Dutch 
news media (Van den Bos, 2018). It concerns a theory assuming a Marxist and/ 
or Jewish network pulling the strings of European institutions — a narrative 
particularly popular on 4chan/pol/. Dutch politicians have subsequently 
flirted with such sweeping theories. For instance, the party Forum voor 
Democratie tweeted that Mark Rutte was a puppet of the Jewish philanthropist 
George Soros,” while the party’s leader Thierry Baudet supported the most 
conspiratorial aspects of the ‘cultuurmarxisme’ debate, tweeting that the 
European Union is ‘a cultural Marxist project aiming to destroy European 
civilization’3 NOS, the public broadcaster, subsequently published an article 
framing George Soros as an ‘influential meddler with tentacles deep in world 
politics’,* which was later withdrawn after heavy criticism pointing out the 


2 The original Dutch tweet by Forum voor Democratie notes: ‘@MinPres [i.e. Mark Rutte] 
draait er niet eens meer omheen: De belangen van NDO’s (lees: Soros) gaan boven het beleid 
van de democratisch gekozen regering van #Hongarije. Hoogste tijd dat deze loopjongen van 
het grootkapitaal nu van het toneel verdwijnt. Reken af met #Rutte op 20 maart! Stem #FVD’ 
(@fvdemocratie, 14 Sep. 2018). 

3 The original Dutch tweet by Baudet noted: ‘Omdat de Europese Unie een cultuurmarxistisch 
project is dat tot doel heeft de vernietiging van de Europese beschaving’ (@thierrybaudet, 19 
Aug. 2017). 

4 Intheir original article, NOS used the title ‘George Soros: invloedrijke bemoeial met tentakels 
ver in de wereldpolitiek’ and noted: ‘De jood Soros steunt organisaties die regeringen openlijk 
bekritiseren [...]. Dat moet stoppen, zeggen tegenstanders’ (Peek 2018). 
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framing’s commonalities with anti-Semitic rhetoric (Peek, 2018). While direct 
ties between such mainstream attention and fringe internet platform rhetoric 
are not to be drawn, each incident in its own right could be situated in the 
aforementioned dynamics of amplification in a Dutch context. 

The falsehoods cooked up on the deep vernacular Web are hard to 
grasp through the concepts of ‘disinformation’ or ‘fake news’, since their 
‘fakeness’ is broader than deliberately coordinated campaigns or clearly 
false information. Rather, they speak to what Muirhead and Rosenblum 
(2019) call a ‘new conspiracism’, whereby sweeping accusations are made 
independent of evidence or coherent explanations, and complex phenomena 
are ‘explained’ through ‘conspiracy without theory’. Such conspiracism is said 
to be dangerous since it delegitimizes the knowledge-making institutions 
at the foundations of democratic societies (Muirhead and Rosenblum, 2019). 
Importantly, this conspiracism can be fuelled by or work alongside a mix of 
foreign interferers, sincere believers and hyperpartisan actors. 

To understand these broad range of actors and interests that stimulate the 
emergence of problematic information, the concept of ‘junk news’ (or ‘pulp- 
nieuws’) is more apt. Junk news shifts the focus from clear and coordinated 
falsehoods towards a broader notion of news crafted to be engaged with and 
to circulate, which, in turn, stimulates polarizing or ‘simple’ information 
that ‘saturates public debate’ (Venturini, 2019). Junk news thereby forms 
an umbrella term for conspiracies, hyperpartisan slander, ‘ironic’ false- 
hoods, low-effort clickbait articles, as well as deliberate disinformation. 
The circulation of these types of junk news has a plethora of reasons, but as 
noted ‘tightly-knit communities’ (Zannettou, 2017) and ‘virality-oriented’ 
subcultures creating and engaging with this highly ‘shareable’ content are 
said to be a crucial factor in their effectiveness (Venturini, 2019). 

Much has been said about the grassroots production of false narratives 
within spaces like Reddit and 4chan (Marwick and Lewis, 2017; Phillips, 2018; 
Tuters et al., 2018; Benkler et al., 2018). However, a more elementary question 
is usually left untouched: what kinds of news sources do these actors rely on 
themselves? Zannettou et al. (2017) found that “fringe” communities often 
succeed in spreading alternative news to mainstream social networks and 
the greater Web’ (1), employing a statistical model (Hawkes process) that 
indicated that fairly marginal spaces like Reddit’s pro-Trump subforum r/ 
The Donald and 4chanss /pol/ board are often first to post a URL to alter- 
native news, only later catching attention on Twitter. They furthermore 
traced which alternative sources were shared on Reddit, 4chan, and Twitter, 
showing that alternative news was shared more often on 4chan/pol/ and 
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select subreddits than on Twitter, noting the popular use of breitbart.com, 
rt.com, infowars.com, and sputniknews.com across the three platforms. 

What about junk news in a Dutch context? Is there a Dutch alternative ‘junk 
news’ network within the deep vernacular web, or do these platforms mostly 
rely on mainstream sources? If found, how vast is the presence of Dutch junk 
news in these spaces? In identifying linked-to websites, can signs of coordinated 
disinformation campaigns be discerned? Or are the types of junk news shared 
mostly hyperpartisan, clickbait, or some other ‘junk’ category? These questions 
are of interest when applied to fringe and ‘extreme’ spaces like 4chan/pol/ but 
can also aid in positioning more widely used yet still largely ‘alternative’ spaces, 
like the largest Dutch subreddit, r/thenetherlands. The research reported here 
thereby begins with the question, where does Dutch junk news appear (if at 
all) on Reddit and 4chan/pol/? Subsequently, it asks, what kinds of junk news 
resonate? It concludes with a brief section on YouTube as a possible alternative 
news network by following the links to Google’s video platform. 


Demarcating the Dutch spaces and Dutch junk news in the deep 
vernacular Web 


Case studies: Reddit and 4chan 


For Dutch cases of virality-oriented subcultures, we focus on Reddit and 4chan/ 
pol/. Although less known than the likes of Twitter, Facebook, Instagram and 
YouTube, Reddit is one of the largest discussion sites globally, with Alexa metrics 
currently showing 234 million unique visitors per month. The platform is divided 
into different subreddits dedicated to the discussion of specific topics, such as 
r/tennis or r/politics. Posts on these subreddits can be ‘upvoted’ or ‘downvoted’ 
by users. The higher the post’s score (upvotes minus downvotes), the higher it is 
placed in a ranked list of content and the more visibility it gains. In the comment 
section underneath every post, ‘redditors’ discuss, debate, or simply joke around. 
Reddit’s Dutch user base seems to be growing (as is shown below), with the 
largest Dutch subreddit r/thenetherlands amounting to 236,000 ‘subscribers’ at 
the time of writing. Its growing popularity makes it an increasingly important 
object of study in a Dutch context. This is heightened by the fact that Reddit 
has been identified as a target of multiple Russian disinformation campaigns, 
with ‘at least a hundred’ IRA accounts influencing the 2016 U.S. elections and 
campaigns continuing into late 2018 (Collins and Russel, 2018; Lagorio-Shafkin, 
2018). Exploring whether such campaigns have also transpired within Dutch 
spheres of Reddit is thus part of the objective of this research. 
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The second case study is 4chan, an infamous imageboard where users post 
anonymously within one ofits subforums (called boards) dedicated to different 
topics like videogames or fitness. 4chan is ephemeral, meaning posts are deleted 
from the site after a few days or even hours. It is a visual environment conducive 
to the production of viral content and generation of junk news (Venturini, 2019). 
The space’s creativity extends beyond the generation of alternative theories, as 
4chan is also infamous as the ‘birthplace of internet memes’, as well as a hotbed 
for nebulous political movements. The latter include ‘Anonymous’, the loose 
‘masked! collective of geeks and hackers infamous for trolling and DDoSing the 
likes of the Church of Scientology and MasterCard (Coleman, 2014), as well as 
more recently the ‘alt-right’, once characterized as an ‘amalgam of conspiracy 
theorists, techno-libertarians, white nationalists, Men’s Rights advocates, 
trolls, anti-feminists, anti-immigration activists, and bored young people’ 
(Marwick and Lewis, 2017: 3) but now arguably pertaining to the extreme side 
of those far-right actors. For this research, we chose to focus squarely on 4chan’s 
politics board, /pol/. This is the most relevant board in relation to the research 
questions, for it is currently among the most active boards on the website? and 
is a fertile ground for conspiracy theories (Tuters et al., 2018) and alternative 
news sources (Zannettou, 2017). 4chan/pol/ is a far-right space, identified as a 
recruitment zone for neo-Nazis (Wendling, 2018) and connected to various acts 
of extreme violence (Hankes and Amend, 2018). This partisanship naturally 
affects the types of news shared on this platform. For balance, other partisan 
areas of the deep vernacular Web were also considered (e.g., 8chan/leftypol/) 
but were ultimately found too insignificant in terms of Dutch activity. 


Tools and timeframe 


In contrast to mainstream platforms like Facebook and Twitter, data from 
Reddit and 4chan are rather accessible. For most of the data collection, we 
used 4CAT (Peeters and Hagen, 2018), a tool developed by the Digital Methods 
Initiative that captures data from a variety of sources, including 4chan/pol/ since 
November 2013. For Reddit, AC AT makes use of the Pushshift API, which allows 
access to an archive of nearly all Reddit posts and comments (Baumgartner, 2018). 

We chose a timeframe from 1 December 2015 up to 1 June 2019, span- 
ning 41/2 years in total. Whereas most other studies in this volume present 


5  Atthe time of writing, the website 4stats.io, which tracks activity on each achan board, lists 
/pol/ and /v/ (video games) as the most active boards, with almost 50 posts per minute and 120 
thousand posts per day (taking the last 4 weeks as a benchmark). These numbers are supported 
by metrics from our own tools (Peeters and Hagen, 2018). 
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Figure 6.1 The frontpage of Reddit (retrieved 11 June 2019) 
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timeframes based on specific events, this long-term timeframe is more 
suitable here for multiple reasons. Firstly, as we will show, the activity in 
relation to junk news posting on these platforms was shown to be fairly 
marginal in comparison to more mainstream social media websites. A larger 
timeframe thereby aids to arrive at patterns in this relatively small stream of 
data. Secondly, to the best of our knowledge, it is the first time the presence of 
questionable Dutch news is researched on these platforms, so it makes sense 
to start with a high-level perspective on the object of study instead of limiting 
it to a particular case. Moreover, this timeframe includes a variety of major 
political events in the Netherlands, including the general elections in 2017 and 
more recently the provincial and European parliamentary elections in 2019. 


Analyses: Haystack to needle and needle to haystack 


As the research focuses on the presence of junk news linked to on Reddit 
and 4chan/pol/, it takes URLs as the primary research objects. To provide 
an overview of the types of news linked to, we decided to focus on domain 
names (sources) instead of links to individual articles (stories). To identify and 
categorize domains, we used two related approaches, referred to metaphorically 
as ‘haystack to needle’ and ‘needle to haystack’. The haystack to needle approach 
denotes a macro to micro inquiry where all domains posted were categorized 
in order to subsequently identify the presence of Dutch junk news within this 
larger pool of data. The needle to haystack does the reverse and starts from 
an expert list of Dutch junk news domains and subsequently enquires into 
when and where these sources appear, and, for Reddit, what kinds of users 
post them. The next two subsections describe these approaches in more depth. 


Haystack to Needle 


The haystack to needle approach moves from a high-level overview to the 
categorization of particular linked-to domains, specifically by parsing (1) 
news from non-news, (2) Dutch news from non-Dutch news, and (3) types 
of junk news (mainstream/junk and types of junk). To do so, a Dutch sphere 
first had to be defined for Reddit and 4chan/pol/ from which an initial list 
of domains could be extracted. For Reddit, the full dataset of opening posts 


6 The expert list is comprised of an original list by De Hoax-Wijzer, edited to remove inactive 
sources, with additional sites added through qualitative analysis by University of Amsterdam 
researchers. 
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was filtered for a list of Dutch subreddits (thus excluding comments; the most 
‘authoritative’ and visible URLs are usually in opening posts). The relevant 
subreddits were compiled from a set of ‘related communities’ posted by 
administrators of r/thenetherlands? and supplemented through querying 
Dutch issues on Reddit. This resulted in a final collection of 182 subreddits 
(see Appendix 6.1). Not all of these subreddits were equally of interest, but 
we nonetheless kept the full list considering our bird’s-eye approach. On 
4chan/pol/, all posts show a flag icon indicating the location of the IP address 
of the poster. To identify a Dutch sphere on /pol/, all posts with a country 
flag of the Netherlands were extracted. It is important to note that this only 
results in a partial sample of Dutch posters, since users can also choose to 
display a custom flag (like ‘Hippie’) instead of one based on geolocation, or 
they can spoof their IP addresses. In all, the dataset collected consists of over 
2 million posts with Dutch country flags, forming a large enough sample to 
gauge the presence of Dutch junk news using the haystack to needle approach. 

Having demarcated Dutch spheres on Reddit and 4chan/pol/, domains 
from URLs posted were extracted from all posts. For Reddit, this resulted 
in 3,489 unique domains. To make categorization manageable, only the 
domains that were posted five times or more were retained. This resulted 
in a list of 372 domains. Similarly, domain names from the Dutch 4chan/ 
pol/ posts were extracted using 4CAT, yielding 8,048 domains. To arrive at 
a comparative sample, we kept the domains that were posted twenty times 
or more, resulting in 352 unique domains. 

The two lists were then categorized according to (1) whether the domains 
were news websites, (2) whether the news websites were in the Dutch lan- 
guage or concerned Dutch affairs, and (3) the category of news websites 
they would fall in. ‘News websites’ here refer to a fairly broad selection of 
websites focusing on the production of news and opinionated columns 
which contain a section dedicated to timely updates. They include blogs on 
current affairs, special interest news, and websites of TV news programmes. 
Thereafter, the news sources were categorized as follows: 

— Mainstream: Reporting by ‘established’ general news outlets with a 
predominantly neutral tone of voice. 

— Other mainstream: All other mainstream news websites concerning 
special interests, such as business or sports news. 


7 See: https://www.reddit.com/r/theNetherlands/wiki/related. Accessed 25 March 2019 

8 This is a higher number than for Reddit because for 4chan, not only the first posts in a 
thread were kept, but also the replies, matching 4chan’s infrastructure of more ‘horizontal’ 
conversational threads. 
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— Disinformation: Sources deliberately publishing false information, often 
with harmful intention, and part of a network or campaign. 

— Hyperpartisan: Extremely coloured and ‘openly ideological’ reporting and 
editorializing from a far end of the political spectrum (Herrman 2016). 

— Clickbait: Sources consisting mainly of articles with sensational head- 
lines and gossip, often in the form of cliff-hangers and listicles, with a 
financial incentive to gain advertising revenue. 

— Conspiracy: Sources mainly dedicated to propagating a range of ex- 
planations to events behind which are secret plots and multiple actor 
entanglements. 


Three researchers categorized the domains, discussed the debatable cases 
with other researchers in this volume for higher intercoder reliability, and 
used external sources like mediabiasfactcheck.com. Mostly, these discus- 
sions were held for websites that could be categorized with multiple labels 
or that fall between hyperpartisan and mainstream such as tendentious 
ones (Peeters and Rogers, this volume). A caveat to this method is that 
categorizing websites on a source instead of story level results in stories being 
labelled, for example, as ‘hyperpartisan’, even though the categorization 
would differ on a story-by-story basis. Websites like The Post Online, for 
instance, contain stories from press agencies as well as tendentious and 
hyperpartisan ones. Despite this, the rigorous domain categorization did 
allow preliminary overviews, which is why it was fitting for the ‘bird’s-eye’ 
perspective of this research. In the haystack to needle approach, we kept 
and categorized the non-Dutch news sources, since they made up a sizable 
percentage of posts, especially on 4chan. Considering these are largely 
Dutch users, it is worth understanding what foreign sites they circulate. To 
show the different categorizations (news or non-news, Dutch or non-Dutch, 
types of news), they were visualized in treemap diagrams using the software 
RAWGraphs (Mauri et al., 2017). 


Needle to Haystack 


Next, the needle to haystack approach was used to analyze the prevalence of 
Dutch junk news in the entirety of Reddit and 4chan/pol/, now by starting 
with a list of URLs that were already identified as questionable. This list was 
constructed by combining an edited list by De Hoax-Wijzer (‘Valse Nieuws- 
sites’, n.d.) with websites found through engagement analysis by researchers 
in this volume (see Appendix 6.2). The list refers to Dutch domains known 
to present news of questionable validity, with an overwhelming partisan 
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tone but also occasionally showing traits of conspiracism. The list was 
coded by the researchers who compiled it and contains the categories 
hyperpartisan, clickbait, conspiracy, and disinformation.9 We fetched all 
the posts containing these domains with 4CAT, resulting in 1714 posts on 
Reddit and 443 on 4chan/pol/. 

We then ‘scoped’ how often junk news appears over time, plotting it as 
histograms. To compare these junk posts to all of Reddit, Google BigQuery 
was used. The total number of posts within subreddits where at least one 
Dutch junk domain appears was fetched to calculate the relative pres- 
ence of junk news. Additionally, the ‘size’ of the Dutch Reddit sphere and 
the entirety of Reddit was retrieved through fetching the total number of 
posts on Dutch subreddits and on Reddit overall. The data was mapped 
as circle-pack diagrams with RAWGraphs. For 4chan/pol/, we used 4CAT 
to fetch all posts (both opening posts and replies) mentioning one of the 
domain names from the expert list in the full timeframe. In order to identify 
temporal trends, the amount of posts with Dutch junk domains was plotted 
per month as histograms. 


Characterizing junk news propagation on Reddit 


Finally, to characterize the kinds of actors propagating Dutch junk news 
and the effectiveness of their activities, various metrics were calculated for 
(further anonymized) junk news posters on Reddit. A similar analysis was 
impossible for 4chan/pol/ owing to the imageboard’s anonymity and lack 
of ‘repurposable’ objects to shine light on the posters. Taking the needle to 
haystack approach, 4CAT and the Pushshift API were used to retrieve all 
posts by Reddit accounts who posted a source from the expert list at least 
twice. The retrieved users were considered ‘junk news propagators’ for the 
purposes of this research. The following metrics were calculated for the 
total corpus as well as for individual users: 
— Subreddits most posted often in. 
— Average score of all posts as indicated by the Pushshift A Pl 
— Average score of posts referring to Dutch junk news domain as indicated 
by the Pushshift API. 


9 It also has the category tendentious-hyperpartisan, which seeks to capture sources like 
The Post Online that have stories from press agencies as well as hyperpartisan columns and 
other contributions that could be described as ‘edgy’, anti-establishment and against political 
correctness (Tuters, this volume). 

10 These scores might slightly differ from their latest number; Pushshift stores it only once, 
i.e., upon encountering the post. 
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— Most linked-to domains. 

— Total posts with domains to Dutch junk domains. 
— Percentage of posts linking to Dutch junk domains. 
— Total posts by user. 


The Reddit users’ pseudonyms were (further) anonymized, since not the 
identity but rather the characteristics of the users is of importance here. 
The first four metrics in the list above were plotted for the whole corpus in 
histograms and circle diagrams, while all metrics were also visualized in 
a matrix for the ten most active Dutch junk news posters, i.e., those who 
linked to the domains from the expert list most often. 


Following and categorizing YouTube links 


YouTube emerged as one of the most popular websites linked to on ‘Dutch 
Reddit’ and ‘Dutch 4chan/pol/’. Since the video platform is often described 
as offering alternative news consumption, we also followed the links to 
YouTube videos in all posts in Dutch subreddits and 4chan/pol/ posts 
with a country flag of the Netherlands. Having collected these links, we 
used 4CAT’s ‘YouTube metadata’ module (in turn using YouTube’s API) 
to retrieve metadata on the videos linked to, such as video title, views, 
and topics. We then plotted the thumbnails of the 1008" videos that were 
linked to most often on image walls with a custom Python script. To 
visualize what types of videos these concerned, we plotted YouTube’s 
‘video categories’ (selected by the uploaders) on top of the image wall. We 
finally ranked the most-linked to YouTube channels, derived from the full 
list of videos linked to on 4chan and Reddit, to gain a grasp of the type 
of video content posted. 


Scoping Dutch junk news 


This section explores the scope of Dutch junk news on both platforms 
under study. We do so by showing the volume of posts linking to one of 
the URLs in the expert list (i.e., the needle to haystack approach). These 
are then compared to the overall volume of (Dutch) posts on Reddit and 
4chan/pol/. 


11 We settled on the peculiar number of 1008 since it would make the image wall adhere to 
the common 18:9 screen aspect ratio. 
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Figure6.3 Totalamount of posts and comments on one of the Dutch subreddits 
(see Appendix 6.1) 


Posts and comments in Dutch subreddits over time 


150.000 


100.000 


2016 2017 2018 2019 


@ POSTS AND COMMENTS m POSTS 
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Scoping junk news on Reddit 


First, as a way to ground the corpus, Figure 6.3 shows the total amount of 
posts made on one of the Dutch subreddits (Appendix 6.1). Just like activity 
on Reddit in general, Dutch activity is increasing: in December 2015 there 
were just over 2,000 posts and comments per month, whereas in January 2019 
this number had grown to 14,000 and seems to be rising. 

Does this increase in activity also mean an increase in Dutch junk news 
linked to on Dutch subreddits? As is evident in Figure 6.4, the amount 
of posts linking to one of the domains from the expert list started at a 
maximum of just eighteen instances in 2016. Two subsequent spikes can 
be observed. The first one, in April to July 2017, speak to the ‘spammy’ 
nature of some areas of Reddit, since one user frequently posted a Dutch 
junk news domain (ninefornews.nl) to an English subreddit. The second 
spike is more varied, however, showing a range of websites like boinnk.nl, 
worldunity.me, and ninefornews.nl. Upon closer inspection, these were 
again posted by a single account, mrthirdeye, the closest one will find 
to a ‘fake news troll’, though its posts received little to no engagement 
(discussed in more detail in section 4). The subsequent dip in November 
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Figure6.4 Frequency of posts linking to Dutch junk news domains on Reddit 
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can be attributed to a content policy change in 2017, possibly leading to 
the banning of this malicious account (Alexander 2017). In subsequent 
months, junk news sharing increased somewhat compared to 2016 but 
remained fairly consistent with around fifty instances per month. An 
increase of posts linking to (at the time) a tendentious-hyperpartisan 
website, The Post Online, can be seen in 2019, although no significant spikes 
during the 2019 Dutch provincial elections and 2019 European elections 
can be discerned. In perspective, these numbers do not seem extremely 
troubling, especially since most posts link to hyperpartisan sources instead 
of outright disinformation (see section 3), and furthermore do not receive 
a lot of engagement (see section 4). 

To further put the scope of Dutch junk news on Reddit in perspective, Fig- 
ures 6.5 to 6.8 contain circle pack diagrams that show its amount compared 
to the entirety of Reddit, as measured in terms of posting activity (excluding 
comments). Figure 6.5 shows the size of all subreddits where a link to a Dutch 
junk news source was shared at least once. The Dutch subreddits are tiny 
in comparison to non-Dutch subreddits (Figure 6.5), given the dominance 
of English-language subreddits on the site. There are a few occasions when 
Dutch junk news was shared on very large subreddits, such as r/viral, r/news, 
and r/worldnews, as well as the infamous pro-Trump subreddit, The_Donald. 
Notably, however, in the Dutch subreddits, the proportion of junk news is 
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Figures 6.7 and 6.8 All Dutch and non-Dutch subreddits where Dutch junk news 
appear compared to the size of all of Reddit. Size of circle 
represents the overall number of posts in that subreddit, and 
colour represents the relative amount of posts with junk news. 
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very low as well. As will be touched on, only occasionally does a subreddit 
have over 5% of its posts linking to Dutch junk news. For the majority of 
subreddits, this figure is less than 1%. 

When zooming in on Dutch subreddits (Figure 6.6), a clearer variation in 
the volume of junk news is observable. For the subreddits where these sources 
appear, they are still in small amounts, with the highest percentages appear- 
ing in r/Forum_Democratie (5,37% of all posts), r/meerderheidnederland 
(5,67%), r/de_thierry (4,93%), r/Duindorp (13%), r/The_Wilders (1,97%), and 
r/FreeDutch (3,04%). Most of these subreddits are related to right-wing 
political parties, ideologies or politicians, such as Geert Wilders or Thierry 
Baudet. These subreddits appear mostly because of the frequent posting of 
links to hyperpartisan websites such as De Dagelijkse Standaard. 

When compared to the overall Dutch sphere on Reddit (Figure 6.6), quite 
a large area of the Dutch subreddits has at least some presence of junk news 
from the expert list. Still, the largest and most mainstream Dutch subreddits 
(r/thenetherlands, r/cirkeltrek, r/Amsterdam) contain a negligible amount. 
Dutch junk news can most notably be seen within already polarized or 
partisan spaces, such as the right-wing subreddits listed above. Larger and 
less partisan subreddits like r/thenetherlands seem fairly immune, likely 
because of a different user base and content moderation. 

Lastly, Figures 6.7 and 6.8 provide a zoomed-out visualization of the 
relative amount of Dutch junk news in the entirety of posts on Reddit. While 
some Dutch junk news appears in a number of both Dutch and non-Dutch 
subreddits, it pales in comparison to the total number of posts in other 
subreddits in the research timeframe. Moreover, even though some Dutch 
junk news appears on a number of large international subreddits (in turn, 
making the sphere appear large), the relative number of appearances of 
Dutch junk news in those subreddits is close to zero. Concluding, then, in 
terms of frequency, links to questionable Dutch-language news sources on 
Reddit is a small issue outside of a few partisan subreddits. 


Scoping junk news on 4chan/pol/ 


4chan’s infrastructure allows less of a comparative approach than that 
of Reddit, but some metrics can shine light on the relative appearance of 
Dutch junk news on /pol/. First, to scope the Dutch sphere, the amount 
of posts with the country flag of the Netherlands is fairly stagnant since 
late 2015 (Figure 6.9). Each month, around 40,000 ‘Dutch flagged’ posts are 
made. The amount increased in March 2017, owing to the Dutch general 
elections. While these numbers are lower in comparison to Dutch users on 
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Figure 6.9 Line graph of posts with Dutch country flags on 4chan/pol/ 
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mainstream platforms like Facebook, they are at times comparable to Reddit, 
and a non-negligible number — a fairly significant insight considering the 
extreme political ideas present on /pol/. It is impossible to tell how many 
individual people these numbers of posts denote, however. 

Despite the frequent Dutch posts on /pol/, the amount of posts linking 
to Dutch junk news is quite low (Figure 6.10). Links to Dutch junk news 
domains appear only around ten times per month. One significant spike 
occurs in March 2017, caused by links mostly to The Post Online and De 
Dagelijkse Standaard, again concerning the general election on March 15. 
Interestingly, a similar spike associated with the elections is absent from 
Reddit. Afterwards, however, the amount of posts linking to Dutch junk news 
drops, remaining low for both the 2019 Dutch provincial elections and the 
2019 European elections. Considering the total amount of posts by Dutch /pol/ 
users (averaging around 40,000 posts per month), the amount of references to 
junk news URLs should be considered negligible. This should not be equated 
with a lack of problematic news content, however, as is discussed below. 


Categories of Dutch junk news 


How sizable ofa role do online news media play within 4chan/pol/ and Red- 
dit? What types of domains are linked to when categorizing news domains 
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Figure 6.10 Frequency of posts linking to Dutch junk news domains on 4chan/pol/ 
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posted in these forums? What types of junk news can we discern? This 
section uses the haystack to needle approach to walk through a number of 
tree maps, each showing a different categorization of the most-linked to 
domains. First, the proportion of news websites is compared to non-news 
domains. Afterwards, the news websites are sorted by Dutch or non-Dutch. 
Finally, the categories of these news sources are outlined and discussed 
(mainstream, hyperpartisan, disinformation, etc.). 

Firstly, Figures 6.11 and 6.12 show which domains from the most-posted 
domain sample are categorized as ‘news’. For Reddit (Figure 6.11), 21,6% of 
all posts on Dutch subreddits refer to ‘news’ websites. Notably, tweedekam- 
er.nl appears 15,694 times, caused by the bot u/kamerstukken-bot posting 
parliamentary texts to the subreddit r/kamerstukken. Removing this bot 
increases the news proportion to 50% — quite a considerable number. 
Other non-news websites include reddit.com itself, often used to host 
images and text, and youtube.com and youtu.be, appearing 951 times 
cumulatively. 

4chan/pol/ paints quite a different news/non-news picture. At 16.6% the 
proportion of links to news websites is lower than Reddit’s 50%. After twitter. 
com and en.wikipedia.org, a staggering 50% of URLs point to YouTube. 
Considering this major presence of Google’s video service, it is further 
scrutinized as an alternative news sphere in section five. 
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Figures 6.13 and 6.14 show the news domains on Reddit and 4chan/pol/, 
respectively, according to their origin (Dutch or non-Dutch). On Reddit, the 
domainsshared on Dutch subreddits are almost exclusively of Dutch origin. 
This is likely due to content moderation in these spaces, requiring posts to 
be specifically about the Netherlands (e.g. on r/thenetherlands). The news 
sources on 4chan/pol/ (Figure 6.4), on the other hand, are predominantly 
from Anglophone sources, such as The Daily Mail, The Guardian, BBC, and 
Reuters. This is fairly unsurprising considering 4chan/pol/’s designation as an 
English language space, unlike the Dutch subreddits. Still, it is worth noting 
that Dutch users on 4chan/pol/ are mostly concerned with English sources 
and are thus more internationally oriented in terms of news propagation 
than users on Dutch subreddits. This also implies foreign news sources 
might significantly influence their news consumption. As shown below, 
English junk news is indeed posted by these Dutch ‘anons’. 

Next, we explore the types of news sources, and if ‘junky’, how they can 
be categorized. Figures 15 and 16 show the categorization of the shared news 
domains as mainstream, other (mainstream), conspiracy, disinformation, 
hyperpartisan, and clickbait, as defined in section 1.3.1. As is evident in the 
visualizations, mainstream or special interest (other mainstream) sources 
make up the largest share of URLs posted on both platforms: 99,6% for 
Reddit and 81% for 4chan/pol/. Despite the frequent characterization of 
pseudonymous spheres like Reddit as ‘alternative’, these results are thus 
somewhat counterintuitive since mainstream sources make up the domi- 
nant proportion links shared. On both sites, NOS.nl is the most linked-to 
news source, meaning the established source is highly relevant. For Reddit 
especially, the lack of problematic content in Dutch spaces is remarkable, 
as in these most-posted domains almost no websites from the expert list 
can be found, save for a few instances of dagelijksestandaard.nl and tpo. 
nl. Indeed, the platform and Dutch users show they seem to be inoculated 
against ‘pulpnieuws’. 

Dutch-flagged posts on 4chan/pol/ show a more problematic, hyperpar- 
tisan nature. Here, 21% of top news domains are ‘junk’, with hyperpartisan 
sources making up most of these. Some of these are foreign state-influenced 
and/or hyperpartisan, such as rt.com and breitbart.com, and others are 
outright extremist, like the neo-Nazi website, The Daily Stormer. As alluded 
to above, Dutch junk news seems to play less of a role here. A few sporadic 
instances of far-right disinformation appeared in the Dutch posts (shown 
in orange), all originating outside of the Netherlands. These include links to 
sputniknews.com, the large Russian news website that has been known to 
propagate disinformation (MacFarquhar, 2016; EUvsDisinfo, 2017), as well 
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as two far-right websites that post a large amount of Islamophobic stories, 
Speisa.com and the Gatestone Institute. In posts linking to Russian sources, 
like rt.com and sputniknews.com, the top URLs are usually referring to 
refugee slander, particularly in Sweden (see Appendix 6.4). It is impossible 
to tell who posted these links considering 4chan’s built-in anonymity, but 
it could potentially point to foreign interference. 


Characteristics of Dutch junk news propagation on Reddit 


What are the characteristics of online actors who share Dutch junk news? 
And are their actions effective? While these questions are nearly impossible 
to answer for 4chan, considering its anonymity, Reddit does afford ‘natively 
digital’ (Rogers, 2013) objects to explore the characteristics and effectiveness 
of junk news propagators. This section will therefore discuss a range of 
metrics and lists concerning Dutch junk news on Reddit. 

298 Reddit accounts were found within the timeframe that linked to 
domains from the expert list of Dutch junk news domains. Out of those, 193 
accounts only posted a Dutch junk news URL once. Only sixteen accounts 
did so ten times or more, meaning there is a long tail of occasional junk news 
posters. When these junk news sources are linked to, they furthermore receive 
a lower score on average than other posts these propagators make (Figure 6.17). 
To reiterate, Reddit scores are created by users’ ‘upvoting’ or ‘downvoting’ a 
post, with a high score meaning a post will move to a higher position on a 
subreddit, thus receiving more visibility. As can be seen in Figure 6.17, posts 
to non-junk news by these propagators outperform posts linking to one of the 
sites in the expert lists, with the propagators’ mean score being 9.8 and the 
mean for their posts linking to a Dutch junk source being 5.6. This is mostly 
caused by automated, ‘spammy’ posts. The median for each of these is 1 and 
overall, 1.24 of the 1.72 of junk news posts have a score of1 or less (72%), meaning 
the Dutch junk news posts receive little visibility and approval on average. 

These low average scores do not mean that junk news stories are totally 
void of success, however. 33 of the 1,761 posts received a score of 50 or more. 
Reddit’s infrastructure stimulates a snowball effect of ‘rich get richer’ posts, 
and some of these even scored higher than 1,000. Zooming in on a URL 
instead of a domain level, it shows that most of these stories are hyperpar- 
tisan of tone. Table 6.1 shows the top three highest-scoring posts on Reddit 
linking to a domain from the expert list. All three best-performing spots are 
‘junky’ and Islamophobic in tone. The first concerns a story by De Dagelijkse 
Standaard on rape and refugees. The second and third are both linking to 
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Figure 6.17 Mean Reddit posts scores by Dutch junk news propagators (users who 


posted a link to a Dutch junk news domain at least twice) as reported 
by Pushshift API 


Mean post scores for Dutch junk news propagators on Reddit 
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Data source: 4CAT and Pushshift; timeframe: 1-Dec-2015 to 01-Jun-2019; bar graph 


Table 6.1 The top 3 best performing posts linking to a Dutch junk domain on Reddit 
subject URL subreddit timestamp score 
Amsterdam Square driver dagelijksestandaard. The Donald 22/08/2017 1811 
(terrorist) before declared nl/2017/08/onthulling- 13:31 
a confused and sick Dutch werkelijke-naam- 
national is now revealed van-de-amserdamse- 
to be KHALID K. from stationsrammer-blijkt- 

Casablanca. The media dus-khalid-karmaoui/ 
cover up doesn’t stop! 
Japan ONLY Admits 27 fenixx.org/2017/05/14/ X The Donald 15/05/2017 1211 
Muslim ‘Refugees’, Two japan-only-admits- 06:02 
Already Arrested For Gang 27-muslim-refugees- 
Rape. two-already-arrested- 
for-gang-rape/ 
Japan ONLY Admits 27 fenixx.org/2017/05/14/ ` CringeAnarchy 10/08/2017 936 
Muslim ‘Refugees’, Two japan-only-admits- 5:13 


Already Arrested For Gang 
Rape. 


27-muslim-refugees- 
two-already-arrested- 
for-gang-rape/ 


Data source: 4CAT and Pushshift. Timeframe: 01-Dec-2015 to 01-Jun-2019 
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Figure 6.18 Subreddits where Dutch junk news domains are most often posted 


Subreddits @ where Dutch junk news appears most often 


Data source: 4CAT and Pushshift; timeframe: from 1-Dec-2015 to 31-Jun-2019; circle pack diagram 


the same story by Fenixx that framed a man who drove a car into a group of 
people at Amsterdam Central Station as a Moroccan terrorist, even though 
he was officially declared as unwell and confused. Interestingly, these 
stories are posted in English-language subreddits, notably the pro-Trump 
r/The Donald and the now-banned r/CringeAnarchy, showing how junk 
news from the Netherlands spreads to foreign spaces. 

In summary, the overall performance of Dutch junk news throughout 
Reddit is fairly weak. Moreover, the high-scoring stories are usually hy- 
perpartisan instead of clear-cut disinformation. Dutch junk news thereby 
can garner considerable engagement on Reddit, but it does not do so on a 
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Figure 6.19 Most linked to Junk news domains on all of Reddit 


Dutch junk domains @ on Reddit 


Data source: 4CAT and Pushshift; timeframe: from 1-Dec-2015 to 1-Jun-2019; circle pack diagram 


regular basis. In this sense, Reddit is more ‘resistant’ to junk news than (for 
example) Facebook is said to be (Burger et al., 2019). 

In which subreddits are Dutch junk news domains posted? Figure 6.18 
shows that r/viral links to most Dutch junk news with 543 instances, but 
much of the prevalence is caused by a single ‘spam’ account, receiving no 
engagement whatsoever. More interestingly, r/Forum_Democratie, the 
unofficial subreddit for the currently the largest party in the Dutch Senate, 
comes in second with 312 posts to junk news sites. Other right-wing partisan 
and hyperpartisan subreddits appear further down the long tail, such as 
r/The Wilders, r/FreeDutch, r/meerderheidnederland, r/The_Donald, and 
r/de_thierry. This is mainly caused by posts on these subreddits linking to 
The Post Online and De Dagelijkse Standaard. 
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According to Figure 6.19, the junk domains that are linked to often are 
mostly the well-known right-wing tendentious and hyperpartisan blogs, 
with The Post Online and De Dagelijkse Standaard ranking on top. The 
‘alternative’ news website NineForNews, which has been host to conspiracies 
and hyperpartisanship (Roermund, 2017), is also amongst the most shared 
domains, but this is mostly due to an automated bot posting links to the 
website (u/ninefornews). As such, most of the shared junk news domains 
can be categorized as hyperpartisan, often with an ‘alternative’, right-wing 
stance. Signs of disinformation or coordinated Russian influence are fairly 
marginal, with Novini appearing 22 times, a website known for pro-Putin 
sentiment (Heck, 2017). As such, from this needle to haystack method, partisan 
and hyperpartisan content is easy to find, but disinformation appears to 
be less of an issue. 

Finally, we highlight the Reddit accounts most active in propagating junk 
news to profile actor types. Table 6.2 shows various metrics on the ten ac- 
counts ranked by the amount of posts linking to one of the domains in the 
expert list. As indicated by total posts and average score, some of the accounts 
post frequently but receive no engagement. Most of these are ‘spam’ accounts 
or automated bots. Interestingly, user 1, the aforementioned u/mrthirdeye68, 
has posted many URLs to Russian and pro-Russian websites as RT.com and 
novini.nl, as well as mainstream sources and hyperpartisan websites like 
Red Ice TV. It is possible that user 1 is a Russian ‘troll’. It received no upvotes, 
however, and only posted links to the obscure subreddit r/viral, meaning it did 
not garner any engagement. As such, it is likely this user is an automated bot, 
or some hybrid. Other bots seem more effective, however. User 2, for instance, 
is the abovementioned ninefornews.nl bot, posting a hundred percent of 
posts to this website in the global news subreddit r/news. Of interest here 
is that user 2 does receive engagement, with a fairly high average post score 
of 570 and a junk news post score of 50. As such, Reddit is at least somewhat 
susceptible to manipulation, depending on the ‘strategy’ of its users. 

In terms of issues, it can be discerned that the most active accounts 
are either concerned with Dutch right-wing parties or topics surrounding 
Ukraine. Despite their frequent linking to junk news websites, the most 
active accounts still link most often to platforms like YouTube and Twitter.” 
A cohort of four right-wing partisans can be observed, who are most active on 
r/Forum_Democratie and frequently link to websites like GeenStijl and The 
Post Online. Most of the accounts actually use the Dutch language, and, upon 


12 Further research might scrutinise what YouTube videos or Tweets are linked to, for instance 
to identify further ‘newsy’ sources or influencers. 
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closer inspection, are also likely Dutch natives. As such we may conclude that 
there is not a significant attempt of foreign accounts meddling with Dutch 
affairs, with the possible exception of the now-banned u/mrthirdeye68. 


YouTube as an alternative news network 


Thus far, this text has handled ‘news’ in the conventional sense of designated 
outlets publishing on current affairs. As discussed in the introduction, 
however, the consumption of both amateur and professional reporting 
increasingly occurs on social media. These modern, alternative ways of 
news consumption cannot be identified when the ‘needle’ is formulated as 
traditional news outlets. As we have identified in section 3, URLs linking 
to YouTube are frequent, especially on 4chan. As discussed elsewhere in 
this volume, the video hosting site is host to various spheres of alternative 
news commentary and opinions, leading Zeynep Tufekci to describe it as 
‘the Great Radicalizer’ (2018). Can we indeed outline an ‘alternative news 
network’ working in tandem with 4chan/pol/ and Reddit? This section briefly 
touches on this question by visualizing and categorizing the most-posted 
videos on 4chan/pol/ and Reddit, as well as the most popular channels. 

Figures 6.20 and 6.22 display the thumbnails of the 1008 most-posted 
YouTube videos within our Dutch Reddit and 4chan/pol/ corpora. Figures 
6.21 and 6.23 show the ‘video categories’ for each of these videos. For the 
top videos on Dutch subreddits, 161 are concerned with ‘People and Blogs’, 
129 with ‘Entertainment’, and 118 with ‘News & Politics’. From this, the 
type of content shared is fairly diverse. 4chan/pol/ is more concentrated 
on news and politics, with 196 videos categorized as such, with ‘People & 
Blogs’ following at 95 and ‘Entertainment’ at 64. 

The number of missing videos for 4chan (the black labels) is notable, com- 
prising almost half of the total, indicating 4chan’s extremism as well as YouTube 
content moderation. The number of deleted videos is visibly less on Reddit. 

If one takes the videos labelled as ‘News & Politics’ as an indicator of a 
‘news source’, as we categorized in the sections above, it becomes possible 
to quantify the role of YouTube as a news source on the two platforms. The 
‘News & Politics’ category comprises 11.7% of the still-online videos for Reddit 
in the sample above, and 19.4% for that of 4chan/pol/. Considering the total 
amount of links to still-online YouTube videos in this timeframe — 7,667 for 
Reddit and 26,635 for 4chan/pol/— one can estimate that around 896 ‘News 
& Politics’ videos were posted on Dutch subreddits and 3,748 on 4chan/pol/ 
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by users with a Dutch flag. Comparing these numbers to those presented in 
section 3, for YouTube news videos would constitute the largest and second 
largest source of news content. On Reddit, they would form the second-largest 
news source, only behind NOS.nl with 1,615 mentions. For 4chan, YouTube is 
by far the largest player in relation to news circulation, since the next most 
popular source, NOS.nl (861 mentions), comprises only one-third of the 
amount YouTube news videos. As such, the role of YouTube as a new player 
in the circulation and consumption of news should not be understated. 

Is this dominant presence of YouTube of great significance in the study of junk 
news? Table 6.3 shows the 25 most-occurring channels from all of the YouTube 
links in our two Dutch corpora. Here, the platforms differ significantly. On 
Reddit, some partisan channels can be discerned, like the one for Forum voor 
Democratie and PVV pers, but the list mostly consists of ‘established’ sources 
like NOS, GeenStijl, and VPRO Zondag met Lubach. On 4chan/pol/, however, 
they are far more extreme and potentially harmful. The most-posted channel 
is SouthFront, dedicated to videos on the Syrian civil war. Below that is Stefan 
Molyneux, a popular Canadian YouTuber who promotes ‘scientific racism’ 
and white supremacist views. Further down the list are (hyper)partisan news 
channels like Fox News as well as the Russian RT and Ruptly. Other far-right 
YouTubers and channels also appear, like Paul Joseph Watson and Rebel Media, 
as well as some left-leaning channels like The Young Turks and VICE. Together, 
the channels referred to by Dutch posters are thus ofa hyperpartisan, sometimes 
with a far-right makeup. As such, YouTube videos on Dutch subreddits seem to 
align with consumption of ‘established’ and ‘traditional’ news media outlets, 
while those on 4chan/pol/ show a highly hyperpartisan and polarized landscape. 


Conclusions 


Despite the frequent characterization of Reddit and 4chan as ‘alternative’ 
zones on the Web, the results presented in this text generally do not reveal a 
large share of alternative news networks spreading disinformation within the 
platforms, at least in a Dutch context. Despite a few instances of pro-Russian 
websites like Novini and one suspicious Reddit account, coordinated campaigns 
of malicious users posting links to disinformation seem largely absent. Dubious 
content can certainly be discerned but compared to overall activity (as shown 
in section two) it should be considered fairly marginal within the spaces we 


13 In reality, these numbers will be somewhat lower because not every YouTube URL points 
to videos (they can also refer to channels), although the vast majority in our corpus does. 
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Figure 6.20 The top 1008 most posted YouTube videos in Dutch subreddits. Black 
labels denote deleted videos/channels. Ranked left to right, top to 
bottom 


Reddit - Dutch subreddits Most linked Youtube videos 1 Dec 2015 — 1 Jun 2019 
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Data source: 4CAT, Pushshift, and YouTube API; image wall 


Figure 6.21 The top 1008 most posted YouTube videos in Dutch subreddits, with 
video categories as an overlay. Black labels denote deleted videos/ 
channels. Ranked left to right, top to bottom 


Reddit - Dutch subreddits ` Mos! linked Youtube videos 1 Dec 2015 — 1 Jun 2019 
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Data source: 4CAT, Pushshift, and YouTube API; image wall 
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Figure 6.22 The top 1008 most posted YouTube videos in achan/pol/in posts with 
a Dutch country flag. Black labels denote deleted videos/channels. 


Ranked left to right, top to bottom 


4Chan /pol/ - Dutch country flags Most linked Youtube videos 1 Dec 2015 — 1 Jun 2019 


Figure 6.23 The top 1008 most posted YouTube videos in 4chan/pol/in posts with 
a Dutch country flag, with video categories as an overlay. Ranked left 
to right, top to bottom. Black labels denote deleted videos/channels 


4Chan /pol/-Outch country flags — Most linked Youtube videos 1 Dec 2015 — 1 Jun 2019 
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Data source: 4CAT and YouTube API; image wall 


206 SAL HAGEN AND EMILIJA JOKUBAUSKAITE 


Table 6.3 The most occurring YouTube channels from all YouTube links posted in 
the Dutch Reddit and 4chan/pol/ samples. Data source: 4CAT, Pushshift, 


and YouTube API. Timeframe: 01-Dec-2015 to 01-Jun-2019 


4chan/pol/ - top 25 most-occurring Reddit - top 25 most-occurring 
channels channels 
channel count channel count channel count channel count 
South Front 191 FOX 10 49 AFC Ajax 476 AT5 38 
Phoenix 
Stefan 177 sandersoni611 46 VitesseTV 269 FvD Meems 37 
Molyneux 
Fox News 156 PewDiePie 4 Forum 143 De Speld 36 
Democratie 
RT 155 PaulJoseph 39 Omroep 107 Football- 32 
Watson PowNed Oranje 
The White 136 Acts17Apolo- 38 Politie 95 LISSAUER 31 
House getics #PRO247 
Ruptly 125 Rebel Media 37 BRAXATORES 82 vpro.nl 30 
Right Side 112 VICE 37 VPRO Zondag 69 DeTelegraaf 30 
Broadcasting met Lubach 
Network 
Omroep 108 Fullwhiskey 36 GeenStijl 67 NOSop3 30 
PowNed 
U.S. 76 VICE News 34 Cafe 51 Hoop Stront 29 
Department Weltschmerz 
of State 
CNN 68 ABCNews 34 NOS 51 PvVvpers 28 
Forum 67 DeroVolk 33 WNL 48 RTLZ 27 
Democratie 
Fox Business 65 _corbettreport 31 Xbox 47 Politie Den 24 
Haag 
The Young 56 TopNotch 43 
Turks 


scrutinized. Reddit seems especially resilient against the circulation of junk 
news. In turn, the characterization as actors within gchan and parts of Reddit as 
influential ‘agenda setters’ should therefore likely be taken with a grain of salt. 

What can be observed, however, are the types of junk news that can be 
characterized as hyperpartisan, especially on 4chan/pol/. This appeared 
mostly through links to popular tendentious and hyperpartisan blogs like 
The Post Online and De Dagelijkse Standaard, but also the more clearly ‘fake’ 
(in the sense of conspiratorial) NineForNews. This right-wing bias is expected 
for 4chan/pol/ due to its infamy as a far-right hub; for Reddit it is more 
notable because we took a politically diverse range of URLs and subreddits 
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as a starting point. While labelling these websites as ‘fake’ is problematic, 
they do indicate a non-negligible presence of polarizing content. Indeed, 
section four showed that the most engaged-with articles from these websites 
often concern topics like migration and Islam, instead of other geopolitical 
events like Russian interference. 

Nonetheless, mainstream sources such as NOS.nl remain popular linked-to 
domains on both Reddit and 4chan/pol/. This is somewhat counterintuitive 
since it has been argued the ‘fringe’ characterization of these pseudonymous 
and anonymous spaces implies their users find knowledge in different 
epistemological drawers. Despite these assumptions, the prevalence of 
mainstream sources shows they have at least some authority within these 
online spaces. It is important to note, however, that we have not considered 
how these mainstream websites are discussed. Likely, domains like NOS. 
nl are considered on Reddit as a trustworthy source, while on 4chan/pol/ 
it might be referenced purely to ridicule it or to portray it as ‘fake news’ 
itself — as is discussed elsewhere in this volume. 

One should furthermore not be blinded by exclusively considering 
websites devoted to reporting on current affairs as the sole source of news, 
as YouTube emerged as a particularly big ‘new’ player in relation to news 
consumption and circulation, especially on 4chan/pol/. On Reddit, ‘News & 
Politics’ videos on YouTube are estimated to form the second-largest source, 
while on 4chan/pol/, they are estimated to strongly outperform any other 
news source. From a brief exploration of the YouTube channels posted on 
both platforms, it seems Dutch Reddit is largely linking to fairly established 
sources, like PowNed, Zondag met Lubach, and NOS, while on 4chan/pol/, 
alternative, hyperpartisan, and problematic information channels emerged, 
like Mike Cernovich and RT. As such, non-Dutch YouTube content might 
have a ‘radicalizing’ role on Dutch users within certain Internet forums. 

Since this report concerns the news sources linked to by actors on Reddit and 
4chan, it does not shed light on the grassroots production of alternative news or 
conspiracies within these spaces. As Tuters et al. (2018) show in relation to the 
Pizzagate conspiracy, the wildest theories can be cooked-up in these spaces 
through a short burst of a ‘butterfly effect’ of ‘bullshit’, unobservable when 
merely considering the prominence of URLs. A more holistic approach, also 
taking into account text and images, could thus aid in further contextualizing 
the current ‘fake news’ debate. Instead of identifying isolated issues of ‘fake 
news’, such broader approaches could tackle the interwoven problematics 
surrounding the circulation of ‘junk news’ (Venturini, 2019) and ‘network 
propaganda’ (Benkler et al., 2018), from the conspiracist mindset of ‘virality- 
oriented subcultures’ to the prevalence of polarizing hyperpartisan content. 
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Appendices 
Appendix 6.1 Compiled list of Dutch Subreddits 


Table 6.4 Compiled list of Dutch subreddits 


ADODenHaag,AjaxAmsterdam,Alkmaar,Aluhoedjes,Amersfoort,Amsterdam,AmsterdamE 
nts, Appiememes,Arnhem,Aruba,Assen,avd,AZAlkmaar,Bassie_en_Adriaan,BeermoneyNL 
,~BeNeLux,Bier,Binnenhof,BitcoinNL,Boeken, Bonaire, BuurmanEnBuurman,CariceVanHoute 
n,CelebsNL,Cirkeltrek,Creatievekoppen,Curacao,de_thierry,de_thierry,DeCorrespondent, 
DeGraafschap,Delain,Delft, Depressie,DeSpeldOfNietDeSpeld,DeStaat, DeStagiair, DeTand 
enborstel,DirkJan, Dordrecht, DoucheGedachten,Doutzenkroes,Drenthe,Duindorp,Dumo 
ulin,Dunglish,dutch, Dutch, DutchBoardgames, DutchComedy,DutchDesign,DutchEnts,Du 
tchFIRE,DutchHipHop,DutchHouse,DutchKeto, DutchMusic,DutchPoetry,DutchProblems, 
DutchSkincare,DutchTech,Eindhoven,Elfstedentocht,Enschede,Epica,Eredivisie,Ethtrader 
nl,FCGroningen,FCTwente,FCUtrecht,Feyenoord,Formule1,Forum_Democratie,FreeDutch 
/Frisia,Frysk,GekkeJongens,Geldzaken,Geschiedenis,Gezellig,Glitterplaatjes,groenlinks,Gr 
oningen,Haarlem,HanzeMemes,HeilzameMeems,Hulpdiensten,ik_ihe,JuridischAdvies,Ka 
merstukken,KatholiekeNederlanden,Kibbeling, KNVB,Koffie,Kut_Doen_Op_Tinder,Kutleve 
n,Kutreclames,LearnDutch,LeCutInsideMan,Leiden,Leraren,LimburgMan,Lowlands,Maast 
richt, MamaAppelsap,Marktplaats, MaxV,Medejongeren,meerderheidnederland,Metal_NL 
‚Motorfietsen, NACBreda,NEC,Nedercringe,Nederporn,NepParlement,netherlands,Neth 
erlandsPics, NietDeSpeld,Nijmegen,NLvsFl,Nuenen,NuJijlnActie,oekraineukraine,ossem 
,otonde,Papgrappen,ParadoxPlaats, PECZwolle,PodcastNed,PokemonGoNL,Poldersocia 
lisme, Politiek, Politiekmemes,PSV,RijmenDichten,RMTK,RodaJC,RomeeStrijd,Rotterdam 
,Saba,SCCambuur,SCHeerenveen, ScoutsNL,SportNL,Spyker, StefanieJoosten,Strips,Stro 
opwafels, StudyInTheNetherlands,SXM, SylvieMeis,Tenenkrommend,The_Klaver,The_Wi 
Iders,TheHague,theNetherlands,theNetherlandsFree,theNetherlandsNature,Tiesto,tokk 
iefeesboek,tokkiefeesboek,TokkieFeesboek,Top2000,TUDelft,TuurlijkisDatEenDing,Twe 
nte,Utrecht,VeganNL,Veluwe,VitesseArnhem,Voetbalnieuws,VraagDerNederlanden,Vra 
agHetAanTonyQuark,Wetenschap,WIDM,WithinTemptation,XboxNederland,Zitkamer,Z 
onderContext,Zwolle 
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Table 6.5 Junk news categorization. Edited and enhanced list originating from 


Hoax-Wijzer. 23 March, 2019 


name domain_name category 
Opiniez opiniez.com hyperpartisan 
Stop de Bankiers stopdebankiers.com hyperpartisan 
t Pallieterke pallieterke.net hyperpartisan 
E.J. Bron ejbron.wordpress.com hyperpartisan 
Dagelijkse Standaard dagelijksestandaard.nl hyperpartisan 
Climategate climategate.nl hyperpartisan 
De Staat van het klimaat destaatvanhet-klimaat.nl hyperpartisan 
JDreport.com jdreport.com hyperpartisan 
tpook.nl tpook.nl clickbait 
Nine for news ninefornews.nl conspiracy 
Daily Paper dailypaper.org hyperpartisan 
Parra parra.nu clickbait 
Viraaltjes viraaltjes.nl clickbait 
about media aboutmedia.nl clickbait 
Martin Vrijland martinvrijland.nl conspiracy 
The Loyalist loyalist.nl conspiracy 
desportgek desportgek.nl clickbait 
Even Delen evendelen.net clickbait 
nietbarkie.nl nietbarkie.nl clickbait 
hardewaarheid.nl hardewaarheid.nl clickbait 
The Post Online tpo.nl tendentious- 
hyperpartisan 
Saltmines.nl saltmines.nl hyperpartisan 
eunmask.wordpress.com eunmask.wordpress.com hyperpartisan 
novini.nl novini.nl hyperpartisan 
niburu.nl niburu.nl conspiracy 
React nieuws reactnieuws.net hyperpartisan 
DMLplus dimplus.nl conspiracy 
martinvrijland.nl martinvrijland.nl conspiracy 
world unity worldunity.me conspiracy 
cultuur onder vuur cultuurondervuur.nu hyperpartisan 
volks nieuws uit Amsterdam noir volksnieuwsuitamsterdamnoir.com conspiracy 


stop pas familie drama stoppasfamiliedrama.blogspot.com conspiracy 


Obed Brinkman obedbrinkman.noblogs.org hyperpartisan 
veren of lood verenoflood.nu hyperpartisan 
De fouten van Rutte defoutenvanvvdrutte.nl hyperpartisan 
Finding voices finding-voices.blogspot.com conspiracy 


ik was in haren ikwasinharen.nl hyperpartisan 
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name domain_name category 
Piet Kei pietkei.nl conspiracy 
bewiseman bewiseman.nl hyperpartisan 


Alternatieve Media Nederland 
Apokalypsnu 

Don Quijotte 
Drimble 

Fenixx 

Hector Reban 
Herstelde Republiek 
Kremlin Troll 
Magilando 

Niburu 

Absolute Duality 
Stan van Houcke 
Stelling 

Tref 

Want to know 
Xandernieuws 
APost 

Best Gezond 
Bewust Nieuws 

Blik Op NOSjournaal 
Bovendien 

Brekend Nieuws 
Dagelijks.nu 
Dagelijkse Krant 

De Stille Waarheid 
Earth Matters 
Ella’ster 

Health Bytes 
Healthwatch 
gezondheidswaakhond 
Leeshetnu 
Lekkerwonen 
LikeMag 

Lijstverse 

Live kijken 
Nieuwsdump 
Not100 
Ongelooflijke Verhalen 
Prankster 
Revolutionair Online 
Snuggerd 

Time 2 Wake Up 


alternatievemedianederland.com 


apokalypsnu.nl 
donquijotte.wordpress.com 
drimble.nl 

fenixx.org 
hectorreban.wordpress.com 


herstelderepubliek.wordpress.com 


kremlintroll.nl 
magilando.wordpress.com 
niburu.co 
nl.absoluteduality.com 
stanvanhoucke.blogspot.com 
stelling.nl 

tref.eu 

wanttoknow.nl 
xandernieuws.punt.nl 
apost.com 

bestgezond.nl 
bewustnieuws.nl 
blikopnosjournaal.blogspot.nl 
bovendien.com 
brekendnieuws.nl 
dagelijks.nu 
dagelijksekrant.nl 
destillewaarheid.nl 
earth-matters.nl 

ellaster.nl 

healthbytes.me 
healthwatch.nu 


leeshetnu.nl 
lekkerwonen.org 
likemag.com 
lijstverse.nl 
livekijken.nl 
nieuws-dump.nl 
not100.nl 
smullen-maar.nl 
prankster.nl 
revolutionaironline.com 
snuggerd.nl 
time2wakeup.me 


hyperpartisan 
conspiracy 
conspiracy 
hyperpartisan 
hyperpartisan 
hyperpartisan 
hyperpartisan 
hyperpartisan 
conspiracy 
conspiracy 
conspiracy 
hyperpartisan 
conspiracy 
hyperpartisan 
conspiracy 
hyperpartisan 
clickbait 
clickbait 
conspiracy 
hyperpartisan 
conspiracy 
conspiracy 
clickbait 
clickbait 
hyperpartisan 
conspiracy 
conspiracy 
conspiracy 
conspiracy 


clickbait 
clickbait 
clickbait 
clickbait 
clickbait 
clickbait 
clickbait 
clickbait 
clickbait 
hyperpartisan 
clickbait 
hyperpartisan 
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name domain_name category 
Tips & Weetjes tipsenweetjes.nl clickbait 
Tis Wat tis-wat.nl clickbait 
TrendBuzz trendbuzz.nl clickbait 
Trendnieuws trendnieuws.nl clickbait 
Trendnova trendnova.nl clickbait 
United-Lightworkers united-lightworkers.be conspiracy 
Vaccinatieraad vaccinatieraad.nl conspiracy 
Viraalpunt viraalpunt.nl clickbait 
Viral Mundo viralmundo.nl clickbait 
Viraaltje Viraaltje.nl clickbait 
Vrouwen Dingen vrouwendingen.com clickbait 
Vrijspreker vrijspreker.nl hyperpartisan 
The Post Online — Politiek politiek.tpo.nl tendentious- 
hyperpartisan 
Erkenbrand erkenbrand.eu hyperpartisan 
Das Kapital daskapital.nl hyperpartisan 
Glop glop.nl hyperpartisan 
Appendix6.3 Metrics on domains shared on Reddit and 4chan/pol/ 


Table 6.6 Metrics for the proportions of news, Dutch news, Dutch junk news, 


and categories in posts on Dutch language subreddits, o1-Dec-2015 to 


01-Jun-2019 


Reddit 
01-12-2015 to 01-06-2019 


Count Percentage 


Positive 
News 5959 
Dutch news 5557 


Dutch junk news 24 


OPs Category 
Negative Percentage 
27594 21.64% Other 
402 93.3% Hyperpartisan 
5935 


0.4% Disinformation 
Clickbait 
Conspiracy 


Mainstream 


5255 89.9% 


580 9.7% 
24 0.4% 
0 0% 
0 0% 
0 0% 
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Table6.7 Metrics for the proportions of news, Dutch news, Dutch junk news, 


and categories in posts on achan/pol/ with a country flag from the 


Netherlands, 01-Dec-2015 to 01-Jun-2019 


Reddit 

01-12-2015 to 01-06-2019 OPs Category Count Percentage 

Positive Negative Percentage Mainstream 10399 71.5% 

News 14541 87301 16.6% Other 1414 9.5% 

Dutch news 3403 11138 23.41% Hyperpartisan 2091 14.4% 

Dutch junk news 2809 11732 19.3% Disinformation 241 1.7% 

Clickbait 45 0.3% 

Conspiracy 351 24% 


Appendix 6.4 Most-posted URLs from posts containing links to RT.com 


and Sputnik on 4chan/pol/ 


Table 6.8 Most occurring URLs from posts containing links to RT.com and Sputnik 


by posts with a Dutch country flag on 4chan/pol/. Derived with 4CAT 


Title URL Amount of 
appearances 

Stabbing death of 15yo schoolboy by https://www.rt.com/news/329243- 9 

‘Arab migrant’ classmate in Sweden sweden-migrant-stabbed-teenager/ 

sparks outrage 

Sweden: Rape Capital of the West https://www.gatestoneinstitute. 9 
org/5195/sweden-rape 

Sweden charges 5 teenage refugees _https://www.rt.com/news/369415- 9 

with beating, gang-raping boy for sweden-refugees-rape-afgan-boy/ 

over an hour 

Belgian prosecutor's office denies https://www.rt.com/news/337276- 8 

terrorist track in murder of guard at ` belgium-nuclear-guard-killed/ 

nuclear centre 

Sex Slave Found Chained in Base- http://speisa.com/modules/articles/ 8 


ment of Immigrant Cafe in Sweden 


index.php/item.3584/sex-slave- 
found-chained-in-basement-of- 
immigrant-cafe-in-sweden.html 


(now offline) 


7 Fake news and the Dutch YouTube 
political debate space 


Marc Tuters' 


Abstract 

Fake news is a contested concept. In the wake of the Trump insurgency, 
it has been reclaimed by “hyperpartisan” news providers as a term of 
derision intended to expose perceived censorship and manipulation in 
the “mainstream media”. As patterns of televisual news consumption have 
shifted over the past several years, YouTube has emerged as a primary 
source for “alternative” views on politics. Current debates have highlighted 
the apparent role of YouTube’s recommendation algorithms in nudging 
viewers towards more extreme perspectives. Against this background, 
this chapter looks at how YouTube’s algorithms frame a Dutch “political 
debate space”. Beginning from Dutch political parties’ YouTube channels, 
we find the existence of an “alternative media ecology” with a distinctly 
partisan political bias, the latter which is resonant with the populist-right 
critique of the mainstream media as the purveyors of “fake news”. 


Keywords: YouTube, hyperpartisan media, right-wing populism, comment 
culture, Forum voor Democratie 


Introduction: YouTube as radicalizing platform 


On 1 February 2019, de Volkskrant and De Correspondent published a 
much-anticipated report on YouTube as a radicalization platform: ‘Leidt 
het algoritme van YouTube je naar extreme content?’ (Translated: Does 
the YouTube algorithm lead you to extreme content?) (Bahare et al., 2019). 


1 The research was undertaken with Camille Godineau, Daniel Jurg, Lieve Keizer, Dana Lamb, 
Aikaterini Mniestri and Ashley Snoei. (Special thanks to Daniel Jurg.) 


Rogers, Richard, and Sabine Niederer (eds), The Politics of Social Media Manipulation. Amsterdam, 
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Drawing on data analysis produced by some of the same authors of this cur- 
rent report, it sought to investigate the extent to which YouTube functioned 
as an engine for online ‘radicalization’ (Tufekci, 2018; Holt, 2017). As these 
and other reports claimed, YouTube appears to be playing a significant 
role in the development of a new antagonistic culture of debate, in which 
an ‘alternative influence network’ is said to have the capacity to shape 
public opinion, especially amongst a demographic of young and politically 
rightward leaning men (Lewis, 2018). Amongst the figures who have risen 
to prominence through this YouTube debate culture, is for example the 
now internationally well-known, Canadian academic psychologist Jordan 
Peterson. Peterson is often viewed as a conservative political figure, even as 
a member of the so-called ‘alt-right’ (Lynskey, 2018). This latter term, which 
stands for ‘alternative right’, gained popularity in the aftermath of the 2016 
US election as a means of describing a seemingly new breed of conservative 
online activism that brought together a diverse array of actors united against 
the perceived hegemony of ‘politically correct’ liberal values, often through 
a jokey and transgressive style (Hawley, 2017; Heikkila, 2017; Nagle, 2017). 
Whilst Peterson has refuted an association with the alt-right, in consulting 
how the YouTube algorithm itself categorizes Peterson it would appear that 
the platform nevertheless still views him in this light. How exactly this 
categorization works is inscrutable to all but the owners of the platform. 
And while it should not be taken as definitive proof of what a given channel 
is about, we can nevertheless assume that YouTube’s categorization does 
reflect some essential aspect of its bottom line, which is to keep the most 
people watching for the longest time possible. 

The present research report uses the same platform-centric categorization 
method as introduced above, applying it to studying the space of Dutch 
parliamentary political debate on YouTube. While initially motivated by 
the question of how this space engages with the issue of ‘fake news’, the 
report however moves away from defining fake news as disinformation 
(which is to say the deliberate manipulation of facts) towards conceiving of 
it in terms of a form of ‘hyperpartisan’ information as produced by ‘openly 
ideological web operations’ (Herrman, 2016). This latter conception of fake 
news is furthermore also resonant with the redefinition of the term as it 
has begun to be appropriated by politicians around the world in order to 
describe news organizations whose coverage they find ‘disagreeable’ (Wardle 
and Derakshan, 2017: 16) — notably by Donald Trump who often refers to 
‘establishment’ media outlets such as CNN and the New York Times as fake 
news (Weisman, 2018). In the European context, where laws such as the 
German Netz DG have been passed at the national level rendering platforms 
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responsible for policing this problem, such critics have framed the attempt 
at regulating fake news as a ‘blueprint’ for state censorship (Wardle and 
Derakshan, 2017: 71). In spite of these controversies, the bewildering issue 
of fake news, entangled as it is together with broader changes in political 
and media spheres at a variety of levels, remains relatively understudied 
outside of the American context — the latter which is in many ways quite 
unique for a variety of factors (Benkler et al., 2018: 381-387). 

Whilst the precise mechanisms of YouTube’s algorithms are unknown, 
what is clear is that they are designed to optimize ‘engagement,’ defined 
in terms of ‘views’ as well as the number of ‘comments’, likes’, and so forth 
(Covington et al., 2016). In recent years, YouTube’s algorithm has been 
critiqued as creating a so-called ‘rabbit-hole effect’ (Holt, 2017), whereby 
the platform’s algorithms, as mentioned above, have been accused of recom- 
mending ever more extreme content, in an effort to keep viewers engaged. 
It has thus been argued that this particular environment has helped to 
draw audiences from the mainstream towards the fringe. Along these lines, 
it has indeed been argued that, on YouTube, ‘far-right ideologies such as 
ethnonationalism and anti-globalism seem to be spreading into subcultural 
spaces in which they were previously absent’ (Marwick and Lewis, 2017: 45). 
Academic researchers exploring this phenomenon have, for instance, found 
that YouTube's ‘recommendation algorithm’ has a history of suggesting 
videos promoting bizarre conspiracy theories to channels with little or 
no political content (Kaiser and Rauchfleisch, 2018). Beyond this current 
‘radicalization’ thesis, for some years new media scholars have observed that 
YouTube appears to multiply extreme perspectives rather than facilitating 
an exchange or dialogue between them - as for instance observed in an 
earlier audience reception study of a polemical documentary produced by 
the Dutch parliamentarian Geert Wilders and published to YouTube (van 
Zoonen et al., 2011). 

We may perhaps want to consider the growth of a new combative and 
conspiratorial culture of debate on YouTube, as documented by these more 
recent YouTube studies, in the context of broader global political shifts that 
have been picking up pace in the latter part of the 2010s, the latter which 
may be referred to under the umbrella term of ‘national populism’ (Eatwell 
and Goodwin, 2018). Referred to as ‘thin ideology’ (Mudde and Kaltwasser, 
2017), populism is characterized by a suspicion of the ‘elite’ as well as a purist 
notion of the ‘general will’ of the true people, the latter which is not necessarily 
equivalent to the democratic electorate (Muller, 2016). Recent new media 
scholarship has convincingly demonstrated how such populist anti-elite 
sentiment translated readily into an embrace of alternative news media, 
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particularly in the US context in which the rise of an ‘alternative partisan 
news system’ is said to have played a crucial role in the last presidential 
election (Benkler et al., 2018). While there exists right and left variants of 
the concept, right-wing populists tend to have an advantage in speaking to 
nationalist issues (Goodwin and Eatwell, 2018). In the analysis of political 
scientists Matthew Goodwin and Roger Eatwell, national populism can 
be characterized by four factors, that they call the ‘four D's’. These are a 
distrust in the liberal ‘establishment’, the destruction of long-held communal 
identity owing to forces of globalization, the relative deprivation as ‘neoliberal’ 
economics leads to a rise in inequality and finally the political de-alignment 
from traditional political parties. Whatever the political valence of national 
populism going forward, Goodwin and Eatwell conclude that these four 
factors are destined to have ‘a powerful effect on the politics of many Western 
countries for many years to come’ (Goodwin and Eatwell, 2018). 


Fakeness and hyperpartisanship 


Thus far the problem of fake news has primarily been studied in the context of 
Anglo-American national populism, specifically the political communication 
surrounding the Brexit referendum and the insurgent Trump campaign and 
subsequent presidency. Furthermore, most current studies of fake news have 
tended to focus on the US context, where institutional trust levels in media 
and in the government are said to be at an all-time low (Edelman, 2018) and 
political polarization stands at an all-time high (boyd, 2017). In that context, 
it has been noted that the standard designation of ‘fakeness’, as a diagnosis to 
be remedied by ‘fact-checking’, fails to acknowledge a much more profound 
epistemological problem. As has long been argued in the literature on the 
sociology of scientific knowledge, ‘facts’ are better understood as products 
of negotiated settlements amongst domain experts (Latour and Woolgar, 
1976). The atmosphere of general suspicion towards expertise that underpins 
the rise of national populism thus poses a fundamental epistemological 
problem. This same general atmosphere of suspicion furthermore works to 
undermine trust in professional media institutions as the arbiters of facts. 
It is argued that this particular context plays into an innate psychological 
tendency to seek out bias-confirming information.” 


2 Indeed, from the social psychology perspective, ‘fake news’ would arguably represent a more 
‘natural’ human preference than ‘facts’, insofar as the former more readily provides support that 
conforms to the ‘moral foundations theory’ of human values (see Haidt, 2012). 
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A leading scholar in the field recently posed the dilemma thusly: in the US, 
somewhere between ‘25 and 3o percent of Americans willingly and intentionally 
pay attention to media outlets that consistently tell that audience what it 
wants to hear, and what that audience wants to hear is often untrue’ (Benkler 
et al., 2018: 367, emphasis added). In the aforementioned context, such scholars 
furthermore suggest that technocratic solutions designed to regulate and censor 
this fake news would be ‘neither feasible nor normatively attractive as they 
would certainly generate heated protest from a large spectrum of the populace’ 
(367). Even in less politically polarized contexts the problem of regulation is 
extremely challenging. It is not isolated cases of fake news that are at issue 
but the larger problem of what these scholars refer to as ‘network propaganda’, 
which constructs ‘materially misleading’ narratives from a tissue of facts (102). 
Because it is extremely difficult to establish ‘ground truth’, reliable technological 
solutions to the problem of fake news are thus unlikely at present (377). 

In light of the former diagnosis, the empirical study below reframes the 
issue of ‘fake news’ in the Dutch-language YouTube space by profiling the 
emergence of a network of channels engaged in political debate and commen- 
tary. It conceptualizes elements of this network as hyperpartisan, in the sense 
that they are ‘openly ideological web operations’ (Hermann, 2016). Whilst 
marginal in comparison to mainstream Dutch news organizations these 
channels nevertheless appear highly engaging, at least from the perspective 
of the YouTube algorithm. As alternative news organizations almost all of 
these channels are unique to YouTube, making them ‘natively digital objects’ 
(Rogers, 2013: 1). The empirical research that follows is thus concerned with 
understanding how these channels work, what their issues are, how they ‘do’ 
Dutch national politics, and how they differ from the mainstream. 


YouTube's ‘related channels’ and Dutch political space 


Following the ‘digital methods’ approach (Rogers, 2013), ‘the discussion that 
proceeds here can be considered as an endeavour to ‘repurpose’ YouTube 
as a research device by thinking along those lines that the platform makes 
available to the public. In particular the approach uses YouTube's ‘related 
channel algorithm as the basis for an analytical method that takes a set of Dutch 
alternative news channels as its primary site of study. As a forewarning, it is 
important to recognize the contrived or ‘artificial conditions’ with which the 
medium frames the object (Rieder et al., 2016: 3). These conditions effectively 
make it impossible for the digital methods researcher to identify where the 
medium ends and where in turn the social begins. Though we do have a sense 
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Figure 7.1 Related channels on YouTube. Table where the top row displays the 
name of each Dutch political party and the columns below each 
of these are the media organizations associated with each party’s 


YouTube channel. 29 March 2019 
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of how some of YouTube’s algorithms work from both the official corporate 
statement (Press, 2019), as well as from attempts by scholars to ‘reverse engineer’ 
or ‘teardown’ the platform (Bessi et al., 2016), the precise functioning is unknown 
and in any case likely to change, thus frustrating the exact reproducibility 
of any of our findings. At any time, YouTube may furthermore suddenly and 
unaccountably change its algorithms, which are in any case invisible to all but 
certain engineers at YouTube. Needless to say, the capriciousness of platforms 
renders the effective control of variables practically impossible. Whilst the 
latter is axiomatic to digital methods it should also be recognized as an inherent 
limitation of the methods as well. For these reasons the present report is thus 
best approached as ‘snapshots’ of a milieu that is constantly in flux. 

The empirical research focuses primarily on repurposing YouTube’s 
‘related channels’ for the purpose of analysis of the Dutch political space. 
In order to delineate what we are here calling the Dutch ‘political debate 
space’ in YouTube, we started from the channels corresponding to the Dutch 
political parties. Since all 13 Dutch national political parties currently in 


3 Note that the Dutch labour party visualized on the far right of the graph did not return any 


related channels. 
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the parliament have a YouTube presence, we used these channels as a ‘seed 
list’, or set of starting points, for the subsequent research. Starting then 
from this seed list the first technique compares all of the channels that 
YouTube classified as related to those of the Dutch political parties. This 
particular approach to categorization in all likelihood involved no human 
oversight; rather, it should be understood as an artefact of how the algorithm 
‘values’ the object, in relation to the aforementioned ‘engagement’ metrics. 
Following the digital methods approach, the analytical gambit here is that 
the channels that YouTube suggests may be treated as a measure of how 
the platform views those parties.* 

The most unusual finding is that the algorithm relates one particular 
channel to almost all parties across the political spectrum: Forum voor 
Democratie (FvD). As a new ‘Eurosceptic’ party with a younger demographic 
than the established nationalist populist Partij Voor de Vrijheid (PVV), FvD 
and its agenda seem to dominate discussion in political debate in a network 
of ‘alternative’ channels discussed below, several of which YouTube relates 
to the parties, most notably ‘TheLvkrijger’. Before going on to discuss these 
alternative news channels in detail, the next most striking finding here is how 
the algorithm seems to organize the political spectrum in relation to different 
‘establishment’ news organizations. One cluster of parties is associated with 
CNN, ABC, NOS5 and another around De Telegraaf, media organizations that 
may be considered as relative liberal and conservative/populist, respectively. 
While it is not necessarily easy to arrange the Dutch political parties on a 
left-right axis — as many smaller parties are more issue-based — it is worthy to 
note how the algorithm groups the Groen Links and Denk parties with centre- 
right and right-of-centre parties. In addition to De Telegraaf, the algorithm 
also relates all of the parties in this latter cluster to alternative Dutch news 
organization: Omroep PowNed, a public radio and TV broadcast renowned 
for its satirical news show, PowNews, which often ridicules politicians with 
provocative questions. In what follows we will categorize Omroep PowNed, 
along with GeenStijl, a blog popular for its similarly abrasive style, as members 
of the established anti-establishment alternative news organizations. 

That the algorithm also relates the parties to a smattering of large Dutch 
commercial and public media channels (WNL, RTL Nieuws, NPO Radio 1, 


4  Oneshould note here that social media use machine learning for predictive consumption in 
which ‘success’ is a measure of how correctly the algorithm predicts what a user will engage with. 
A well-known critique here is the notion of the ‘filter bubble’ (Pariser, 2011), which argues that 
algorithmic categorization can have the effect of narrowing the range of alternate viewpoints 
that one is exposed to. 

5 Note that we removed most US channels from Dutch media network visualization below. 
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Figure 7.2 TheLvkrijger post: Translated into English: ‘He who is silent agrees! 
Don't shut up anymore! This is your country! Claim it!’ 
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Veronica Inside), is unsurprising as these would be an expected part of an 
average Dutch media diet. What is likely surprising to those unfamiliar with the 
Dutch political space in YouTube is the network of alternative or ‘alt’ channels 
that YouTube relates to the parties, notably the aforementioned ‘TheLvkrijger’, 
but also ‘Arnews’, ‘Leukste YouTube fragmenten’, ‘Lissauer’ and ‘Rafiek de 
Bruin. With the possible exception of ‘Arnews’, all of these channels could be 
categorized as ‘openly ideological web operations’. As we will see, these Dutch 
political debate channels are ‘natively digital objects [...] “born” in the new 
medium’ (Rogers, 2013: 19), as opposed, for example to Omroep PowNed. While 
some of these channels, like TheLvkrijger, are transparently partisan, national 
populist sentiments seem common in this space, as for example captured ina 
post by TheLvkrijger encouraging viewers to vote in the upcoming elections, 
which featured the slogan ‘He who is silent agrees! This is your country! Claim it’ 


The Dutch YouTube media sphere 


In an effort to create a panoramic graph of the larger Dutch YouTube media 
sphere that would also remain connected to the Dutch political sphere on 
the platform we used YouTube's related channels algorithm to ‘snowball’ out 
from the seed list of the 13 parties to 3 degrees of relations. We subsequently 
visualized the related channel network with network analysis software, 
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Figure7.3 Related channels on YouTube. Panoramic graph of larger Dutch 
YouTube media sphere. This graph was produced two months apart on 
29 March 2019 and again on 22 May 2019 with identical outcomes. 
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where nodes represent channels and edges represent relations according 
to YouTube’s algorithm. The size of the text represents a measure of their 
relative importance within the network. Finally, relative similarity between 
channels determines their colouration, clusters which we have then labelled 
as government, military, commercial, vlog, public, sport and, finally, our 
specific alternative media ecology. The largest nodes in the graph are all 
‘establishment’ media organizations with NOS Jeugdjournaal, RTL Nieuws 
and De Telegraaf at the centre. Slightly outside of the centre another large 
node is the established, anti-establishment channel Omroep PowNed, known 
once again for its ‘edgy’ confrontational style of reportage. If one continues 
along this same line one encounters the cluster labelled alternative media 
ecology at the centre of which the most connected node is FvD but which 
also includes a few government channels (for example Eerste Kamer) as well 
as anumber of the aforementioned ‘alt’ political debate channels which we 
encountered earlier (for example ‘Leukste YouTube Fragmenten’). In the 
next steps of the analysis we will delve more deeply into these ‘alt’ debate 
channels by performing some qualitative analyses of their content. 

In both the panoramic map as well as in the prior analysis (based on only 
a single degree of relations to the seed list), we find the presence of a number 
of ‘natively digital’ political debate channels, such as ‘Leukste YouTube 
Fragmenten’ and TheLvkrijger. In considering these channels as a type of 
mini-genre, we can thus compare their style and how they ‘do’ Dutch politics. 
At the outset it should be noted that, by certain measures, some of these 
channels appear quite marginal. ‘TheLvkrijger’, for example, which YouTube 
related to half the parties, only has 6.5 thousand subscribers. CNN, which 
YouTube also related to half the parties, has 6.5 million subscribers. The Dutch 
political space on YouTube is not that large, however, and in any case, despite 
differing by orders of magnitude, YouTube related channels algorithm places 
CNN and ‘TheLvkrijger’ on the same footing. One degree of relations gives us 
a collection of ‘alt’ political debate channels including ‘TheLvkrijger’, ‘Leukste 
YouTube Fragmenten’, ‘Rafiek de Bruin’, ‘LISSAUER.COM;, ‘Res Cogitans’, 
'Omroep PowNed’, ‘Arnews’, to which we can add a few more by exploring 
their relations including ‘GeenStijl’, ‘AllePolitiek’ and ‘Deweycheatumnhowe’. 
In analyzing their style, we can observe that ‘TheLvkrijger’, ‘Leukste YouTube 
Fragmenten’, ‘Rafiek de Bruin’, ‘AllePolitiek’ and ‘Deweycheatumnhowe'’ are 
all of a sort, in that all post debate clips or interviews. Furthermore, sites 
as ‘Arnews’ and ‘LISSAUER’ use ‘meme’ graphics — a style also employed, 
and in fact pioneered to an extent, by PowNed and GeenStijl. Somewhat 
like Omroep PowNed in style, GeenStijl is famed for its provocative anti-PC 
tone. Settled in the Dutch media landscape (and with PowNed receiving 
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structural funds from the government), they can thus fairly be labelled as 
“established anti-establishment'. Using clickbait tactics to attract attention, 
with the notable exception of AllePolitiek, the aim of these channels seems 
to be to amplify dissensus in the Dutch political space. Whilst this of course 
stands in marked contrast to the country’s long history of consensus politics, 
where one normatively stands on this depends on one’s democratic political 
theory. Furthermore, whilst several channels are transparently partisan, 
what is remarkable is that the majority of the most viewed videos in most 
of the channels focuses on figures from the FvD and PVV. 

To provide a synoptic view of the natively digital debate channels’ issues one 
can look at the most commonly used words in the titles of all of the channels 
in the form of ‘word clouds’ with words colour-coded and sized by frequency. 
Those appearing in black are issues such as referendum, climate agreement, 
dividend tax and Brexit, whist those in colour are the names of parties and 
their spokespeople. At first glance, what one notices is that ‘Arnews’ and 
‘AllePolitiek’ appear primarily issue driven, whilst the other channels seem 
more engaged with Dutch political personalities. One can also observe the 
relative similarity between ‘ResCogitans’ and ‘Leukste YouTube Fragmenten’, 
as channels that both appear partisan towards FvD — on closer inspection this 
is indeed the case (and in fact they even appear to be run by the same person). 
Similarly, ‘TheLvkrijger’ appears to be partisan towards the PVV, which is also 
the case on closer inspection. As with the thumbnails, discussed above, the 
names of the figures from both these parties commonly appear in all these 
channels video titles. Further scrutiny reveals all of these channels to be at 
least somewhat sensationalistic, with ‘Arnews’, often using terms like heated 
debate (‘verhit debat’) in order to describe content. The more partisan of the 
channels follow an antagonistic logic when commenting on parliamentary 
debates, identifying the winner or loser of a given debate, at times resembling 
a debate genre familiar on YouTube, for example in videos featuring Jordan 
Peterson, often labelled in the style: Jordan Peterson DESTROYS so and so. 

Alongside the related channels findings, the fact that official Dutch 
parliament channels, along with Forum voor Democratie (but not the other 
Dutch political parties), are clustered alongside these ‘alt’ debate channels 
seems peculiar. Given the aforementioned capriciousness of platforms, 
might these findings be attributable to an excited algorithm in the aftermath 
of FvD’s surprising success in the senate elections? If so, then one would 
expect these findings to differ when reproduced at another point in time, 
either revealing an underlying stable state of network composition or else 
another excited state. With this question in mind we reproduced these first 
two methods, that were initially explored prior to the provincial (senate) 
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Figure 7.5 Screenshot from the ‘About’ page on Cafe Weltschmertz's YouTube 
channel which includes a sarcastic ‘trigger warning’ for viewers who 
might be angered by its frank approach to political debate, as well as 
crypto-normative espousal of ‘democratic hygiene processes” 


© Cafe Weltschmerz 


election, at the time of the EU parliamentary elections. Remarkably, we found 
no substantial difference in either the channels that YouTube considered 
asrelated to the parties (see Appendix 7.1). Moreover, the panoramic graph 
remained ¿dentical,° suggesting that it may thus reflect an underlying stable 
state of how the algorithm currently categorizes the larger Dutch YouTube 
media sphere (see Figure 7.3). Because the EU elections did involve several 
other parties, we did however identify the presence of two new clusters in 
the panoramic graph: one of which, associated with the new pan-European 
Volt party, floats on its own completely disconnected from the overall 
network; and another, associated with Dutch Pirate Party, which is con- 
nected to the larger network via a channel ‘talking-head chat show’ called 
‘Cafe Weltschmertz’. In close proximity to the alternative media ecosystem 
discussed above, Cafe Weltschmertz seems to frame its political debates in a 
tendentious style similar to some of the channels profiled above — referring 
to its approach, for example, as ‘politically incorrect’. In this same cluster 
we also however find leftist investigative journalism channels including 
‘Follow the Money’ and De Correspondent as well as the expected channels 
focused on the issue of privacy, ‘Bits of Freedom, ‘Privacy First’ and finally a 
debate channel called ‘Potkaars Podcast’ featuring a video on its front page, 
entitled ‘Potkaars praat met iedereen’ (Potkaars speaks with everybody). 
In light of our subsequent discussion of fake news as a topic of debate, the 
videos description is worth quoting at length: ‘If you want real news, you 
have to cut through the smoke -smoke & mirrors- to get to information and 
demand a controllable government. Dismissing information as ‘fake news’ 
is easy. But what do you replace it with?’ 


6 YouTube disabled the related video feature shortly after we completed this analysis (YouTube, 
2019). 
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The final analysis concerns how channels in the Dutch political space ‘do’ the 
one issue in particular: fake news. We begin with a video from ‘TheLvkrijger’ 
of PVV representative Martin Bosma confronting the government minister 
of Internal Affairs, Kajsa Ollongren in a Tweede Kamer debate on the fake 
news that became central to her portfolio. In the video Bosma accuses 
Ollongren of ‘playing a strange game’ with ‘what is truthful and what is 
not’. Bosma points to a fundamental lack of consensus of what's at issue in 
the fake news controversy more generally as well as alleging that Ollongren 
has seemingly tended to change her own definition of what constitutes fake 
news in order to suit her political purposes. When examining the comment 
section below this video we see commenters echoing Bosma’s sentiments 
and questioning Ollongren’s integrity, expressing the need for a concrete 
definition of fake news (45 likes). Commenters furthermore speak of Dutch 
public broadcasting as fake news that does ‘nothing but mislead citizens’ 
(78 likes).® This latter use of the concept of fake news echoes Trump's use 
of the term as means of attacking the establishment media. 

Another video of interest, also published by ‘TheLvkrijger’, features a 
PVV-organized populist-type debate with pundits on the topic of fake news 
(‘nepnieuws’) and the European Union. Similar to the aforementioned 
Trumpian framing of fake news, the debate discusses the supposedly left- 
wing bias in the establishment media, as represented in one participant’s 
statement that ‘media serve the ideology of the establishment’. Again, we 
see positive reception in the comment section where a commenter writes 
about the Dutch public broadcaster ‘NOS = FAKE NEWS’, and advocates 
viewers to seek their news from alternative sources on YT.9 

In another video on the topic, this time published by GeenStijl, a reporter 
asks politicians leaving the Tweede Kamer about the issue of ‘fake news’. 
This time the reporter’s questioning revolves around proposed European 


7 Without offering any analysis of this particular unique term, for reasons of brevity and 
focus, it is nevertheless worth noting here that one of the signature accomplishments of some 
of the American alternative partisan news system, especially those on the far-right, has been 
to introduce new terminology in the hopes of normalizing certain formerly radical conceptual 
frames (Hatewatch Staff, 2015; Benkler et al., 2018: 128-132). In political punditry this technique 
is sometimes called ‘opening the Overton window’ (Marwick and Lewis, 2017: 11) 

8 The number of likes on a comment can be treated here as a measure of agreement with 
these sentiments expressed therein. 

9 This theme of framing of ‘NOS is fake news’ and ‘NOS is left-wing propaganda’ came up in 
multiple comments of multiple videos. 
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Figure7.6 Weighted word lists of the titles of all the videos from the political 
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Figure 7.7 Screenshot of a comment under the video of ‘Leukste YT Fragmenten; 
referring to a‘hopeless debate’ and the lack of consensus on the 


definition of ‘nepnieuws’ 
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legislation, rather than Ollongren’s engagement with the issue. As per the 
channel's provocative style, the video does not hide its partisan stance on the 
issue, titling the video: ‘Brussels is censoring free speech’. Again, representa- 
tive Bosma appears, this time with an attack on liberal political correctness 
emanating from the liberal technocrats in Brussels, stating ‘everything 
that is not politically correct will be tackled’.”° By contrast other politicians 
interviewed by the journalist see the necessity of government action in 
response to the ‘crisis’ of fake news. In the comments section multiple com- 
menters reiterate the theme of the Dutch Government itself being ‘fake news’. 

A video published on ‘Leukste YouTube Fragmenten’ features a Tweede 
Kamer debate fragment, once again on the concept of freedom of speech, this 
time by FvD leader Thierry Baudet. In this clip Baudet makes a sophisticated 
conceptual point on the alethiological (the study of truth). Using logic, Baudet 
tries to refute Ollongren’s concept of fake news as fallacious. He argues that 
if for an atheist god is not true, then that would not make preaching a form 
of disinformation. Based on this argument he then claims that Ollongren 
would ‘accuse the teachings of Catholicism of being untrue’ and thus ‘a form 
of disinformation’. After his sophistry, Baudet then goes on to make the point 
that state actors should not be allowed to decide what is true and what is 
not true. ‘You cannot trust the state’, he says, what ‘we need’, he argues is 
‘free press’. In the comments section commenters state that all politicians, 
besides Baudet, define fake news subjectively in particular falling back on the 
Russian ‘evil actor’ narrative, which a commenter characterized as ‘Orwellian’. 

Although our analysis in the report did not include any left-of-centre 
Dutch political commentators, this is not to say that they do not exist on 
YouTube. Rather, the methods we used did not bring them to the fore. Indeed, 
alongside the ‘alt’ channels profiled above we can in fact find a video of Arjen 
Lubach’s Zondag met Lubach, the VPRO broadcast in which the commentator, 


10 Political correctness is a very popular straw man amongst ‘dark intellectual web’ figures 
like Jordan Peterson on the right (Weiss 2018), but also left-wing figures such as Slavoj Zizek. 
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as with the one on the Green Style video, critiques the Russian “evil actor' 
narrative. In Lubach's opinion the real threat is in fact an alt-right conspiracy 
theory, in the style of Pizzagate, which Russian actors merely amplify. 


Conclusions: Left-leaning bias? 


Academics are often accused by conservatives of having a left-leaning 
bias; indeed, apparently evidence reflects these allegations (Abrams, 2016). 
This narrative of ‘liberal bias’ has been one of the central themes of the 
American new right, extending from contemporary ‘neo-reactionaries’ 
(Malice 2019), to 1990’s ‘culture warriors’ (Nagle 2017), and back to the 1960's 
‘messengers of the right’, who pioneered new media formats in order to 
disseminate their message (Hemmer 2016). And whilst accusations of such 
perceived liberal bias may be offered against this report, the fact remains 
that we came by the data underlying our findings by merely following 
the platform and the way that it categorized the Dutch political parties. 
In doing so we identified a series of ‘alternative’ debate channels many of 
which appeared hyperpartisan — following Hermann’s initial definition of 
the concept as ‘openly ideological web operations’. If we were to locate the 
political bias of these ‘alternative’ political debate channels in relation to 
‘establishment’ media organizations in the Netherlands, then many would 
seem to be roughly aligned with the conservative and populist tone of De 
Telegraaf. Closer still to the antagonistic debate style that we observed in 
many of these channels is the transgressive style of reportage pioneered 
by the ‘established anti-establishment’ of GeenStijl and Omroep PowNed. 
The Netherlands is also well known for having innovated new media formats, 
notably reality TV. Additionally, one might also say that the Netherlands has 
been innovative in developing new positions and issues on the right — notably 
the issue of homo-nationalism (Aydemir 2011). What we may however also 
be seeing in this research is the possible emergence of US-style right-wing 
punditry in the Dutch sphere. While it still seems marginal in the current 
‘alternative’ debate space on YouTube space, exemplary here is the channel 
of ‘Paul Nielsen’ (24,531 subscriptions), an English language Dutch ‘alt-lite’ 
channel which features such titles as: ‘NOS is the Dutch CNN | Biased News in 
Holland’ and ‘How Marxists took over the Netherlands’. The site claims to be 
endorsed by Prof. Dr. Paul Cliteur, expert witness at Geert Wilders’ hate speech 
trial and Ph.D. supervisor to Thierry Baudet. This channel may be a bridging 
node to the figures in what has been called YouTube’s ‘dark intellectual web’ 
(Weiss, 2018) or its ‘alternative influence network’ (Lewis, 2018), such as for 
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example Stephan Molyneux who features a video with the title: ‘The Truth 
About Immigration and Crime in the NL. At the same time, in scrutinizing 
a network one should be careful of the guilt by association fallacy. The point 
is rather to acknowledge the proximity to an active and controversial area of 
debate within the platform. 

While the possible intervention of ‘Russian trolls’ as a factor in 2016 US 
elections has been convincingly made (Jamieson, 2018), the Dutch case is 
different. In addition to the absence of an Anglo-American ‘first-past-the-post’ 
electoral system there is a very different media ecosystem in the Netherlands, 
which for example still has a much higher trust in the general ‘establishment’ 
than in the US (Edelman, 2018). Furthermore, as opposed to the ‘neutrality’ 
axiom that has characterized 20'"-century US news media, Dutch news media 
have always been partisan. This having been said what we see in YouTube 
suggests the emergence of a hyperpartisan Dutch new media political space. 
Currently it is mostly dominated by one party, but other parties may take this 
as a challenge. Insofar as YouTube represents a media source in the Nether- 
lands, especially for youth, the Dutch YouTube ‘alt’ political debate space may 
represent a re-politicization of youth, which runs counter to neoliberalism’s 
historical project of pre-emptive depoliticization (Foucault, 2008). If political 
pluralism advocates peaceful coexistence of different interests the combative 
and anti-politically correct tone of much of political debate on YouTube may 
militate against this. Can the long tradition of consensus in Dutch culture be 
brought to bear on this new debate culture or is the Netherlands on the path to 
Americanized Trump-style polarization? In terms of final takeaways, we can 
say that an inquiry into fake news, which defines the latter as the deliberate 
manipulations of facts, must also consider the inherently problematic aspects 
of this very conception as well. For this reason, regulating disinformation can 
be portrayed as Orwellian ‘thought control’, which in turn resonates with 
populists’ anti-establishment, conspiratorial frameworks. 
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Figure 7.8 Related channels on YouTube, 22 May 2019 
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LIST OF MOST IMPORTANT NEWS ORGANIZATIONS (RELATEO CHANNELS) 


Table where the top row displays the name of each Dutch political party 
who ran candidates in the EU election. As with Figure 7.1, the columns 
below each of these are the media organizations associated with each 
party’s YouTube channel. The related channels for the parties are identical 
to Figure 7.1 apart from a few minor differences and the fact that D66 now 
no longer returns any related channels, as with PvdA. Note also that of the 
two EU parties that return channels are categorized quite differently than 
the other national Dutch political parties. Source: YouTube. 


8 Conclusions 
Mainstream under fire 


Richard Rogers and Sabine Niederer 


Abstract 

To what extent do (foreign) disinformation and so-called fake news 
resonate in political spaces online within social media around the 2019 
provincial elections and the European parliamentary elections in the 
Netherlands? We found no foreign disinformation, fake advocacy groups 
or imposter news organizations, but we did take notice of a polarised 
media landscape, where problematic information, including extreme 
content, is engaged with (liked, shared, retweeted, etc.) or returned in 
search engines when querying political parties, political leaders as well as 
social issues. The study ultimately recommends media training as well as 
disengagement with extreme content, together with a call for continued 
access to social media platform data for media monitoring purposes. 


Keywords: fake news, disinformation, polarisation, extreme content, 
media monitoring 


Separating disinformation and fake news and developing other 
notions further 


Disinformation and fake news are contemporary phenomena with rich 
histories. Disinformation, or the wilful introduction of false information 
for the purposes of causing harm, recalls infamous foreign interference 
operations in national media systems, such as the Russian campaign 
‘Operation Infektion’ that in the early 1980s effectively publicly linked the 
HIV virus with a supposed, secret US bioweapons lab. Outcries over fake 
news, or dubious stories that have the trappings of news, have occurred 
repeatedly with the introduction of new media technologies that disrupt 
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the publication, distribution and consumption of news — from the so-called 
rumour-mongering broadsheets centuries ago to the blogosphere more 
recently. Social media are only the most recent ‘truthless’ media. Designating 
a news organization as fake, or calling it der Liigenpresse, however, has a 
darker history, associated with authoritarian regimes or populist bombast 
diminishing the reputation of ‘elite media’ and the value of inconvenient 
truths more generally. 

These days social media platforms have been implicated in both the 
practice of disinformation as well as the rise of these two varieties of fake 
news. As discussed in the theoretical and empirical scholarship to date, 
social media have enabled the penetration of foreign disinformation opera- 
tions, the widespread publication and spread of dubious content as well as 
extreme commentators with considerable followings attacking mainstream 
media as fake. 

Worldwide, disinformation and fake news are increasingly under study 
together, but the argument could be made to separate them. Indeed, in the 
Netherlands evidence of foreign disinformation campaigning is scant; cases 
of domestic actors employing the ‘Russian playbook’ of disinformation tactics 
are also seldom documented. Unlike in the case of the US, to which much of 
the scholarship is dedicated, in the Netherlands one has not witnessed the 
rise of imposter news organizations or the formation of advocacy groups 
purporting to represent social groups or causes. Indeed, when employing 
narrow definitions of disinformation and fake news, there is hardly any to 
be found in the Netherlands. 

But definitions of fake and its next-of-kin ‘junk’ news often extend 
to clickbait, conspiracy, hyperpartisan and tendentious sources as well 
as artificially amplified social media content and accounts. As a case in 
point, when Buzzfeed News famously reported in 2016 that ‘fake news’ was 
outperforming mainstream news on Facebook, included in its definition 
were clickbait and hyperpartisan sources, such as Breitbart News. Expand- 
ing the definition in such a manner would have consequences in that the 
Netherlands has all of them in relative abundance. 

Initial studies have found that the Dutch are great consumers of clickbait 
and ‘pulp’ content; there is a well engaged-with set of tendentious and highly 
partisan news-like organizations especially on the right of the political 
spectrum, and the artificial amplification of social media accounts, including 
those of certain politicians and musicians, has been well documented. 
Their sway varies. Clickbait is said to be consumed more often than main- 
stream news, though there is also more of it. Conspiracy is perhaps the 
least clicked on, at least according to our findings per platform, discussed 
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below. In political spaces online, news furnished by commercial and public 
broadcasting are still referenced, liked or shared in greater quantities than 
tendentious and hyperpartisan sources, though the latter has been present 
in the most engaged-with lists of sources around election issues. Artificial 
amplification both burnishes one’s image but also has led to mini-scandals 
when fake followers are revealed through new online detection tools and 
news reporting. 

Whether any of them is particularly persuasive is a question increasingly 
posed. The first wave of scholarship on the production and spread of disinfor- 
mation has yielded to the next wave on its effects. Given people’s hardened 
attitudes the question concerns whether the influence of disinformation 
and fake news is ever more than ‘minimal’. 

In that regard, the rise of extreme content (including extreme clickbait), 
circulated on social media platforms, is one source of continuing con- 
sternation and measurement, leading to calls for platform regulation and 
prompting social media companies to hire more content reviewers and work 
on automated detection. Another source of concern is the mainstreaming 
of doubt and trust in public institutions and media, concomitant with the 
rise of both ‘alternative facts’ and ‘alternative fact infrastructures’. The 
post-truth condition, as it is termed, is discussed as both first-order ‘fact 
fights’ as well as second-order competitions between ‘sectarian knowledge’ 
regimes and competing media ecologies. Is the authority of mainstream 
news and knowledge institutions declining for increasing segments of 
society that consume the alternatives? One finding often related is that 
older consumers are ‘available audiences’ for fringe media and are relatively 
‘heavy users’. 

The consuming and sharing of so-called fake news have been the subject 
of media literacy initiatives, including quizzes, serious games and public 
service campaigns. Through heightened awareness, especially around 
the time of elections, the impact on consumers of any disinformation and 
dubious content may be mitigated and the institutions made resilient, it has 
been argued. Voluntary and professional fact-checking are also discussed in 
this regard, as are (automated) content flagging, together with the need for 
human review. The question regularly posed is whether the sheer amount 
of platform junk will overwhelm the capacity to review it, together with 
the related issue of who will review the reviewers. 

Finally, there is widespread scholarly concern about the restrictions to 
public access of social media data, otherwise known as the issue of ‘locked 
platforms’. Future research should address the extent to which disinforma- 
tion and ‘fake news’ (in all its more exacting definitions) continue to thrive 
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online, and whether there are monitoring capacities in place so that its 
overall consumption and persuasive capacity may be measured, and the 
wider societal implications may be studied and acted upon. 


Empirical findings concerning junk news around the Dutch 
elections of 2019 


The present study consists of a series of empirical case studies concerning 
the engagement with junk news, including hyperpartisan and tendentious 
sources, in Dutch political spaces in social media in the run up to two elec- 
tions in 2019. These spaces were demarcated using queries of politicians’ and 
political party names as well as social issues, some related to the elections 
(such as climate and EU) and others more controversial (such as Zwarte Piet). 
Here the findings are summarized, and subsequently put into perspective 
in a discussion of their implications, also for policy. 

The overall research question driving the study is as follows. To what extent 
do disinformation and so-called fake or junk news resonate in political spaces 
online within social media (and search engine returns) around the 2019 provin- 
cial elections and the European parliamentary elections in the Netherlands? 

Here the findings are provided in brief: 

1) We found neither foreign disinformation (campaigning) nor fake ad- 
vocacy groups operating around the Dutch provincial and European 
parliamentary elections of 2019. 

2) Mainstream news is consumed and engaged-with more than junk 
news, but not for all platforms in all periods under study (8 Febru- 
ary-25 March 2019, 26 April-24 May 2019 or longer durations). The issue 
spaces around Zwarte Piet and MH17 have proportionately higher 
quantities of junk news than election issues and are also ‘year-round’ 
issues, so to speak, rather than seasonal or event-based. 

3) With respect to social media manipulation, troll-like users are active 
across Dutch political issues spaces (on Twitter). We also suspect there 
is (rather light) artificial amplification taking place (fake followers on 
Instagram). 

4) There is the emergence of a hyperpartisan/tendentious alternative media 
ecology, competing with the mainstream news and also mainstreaming, 
in the sense that these sources include newswire content and are spread 
by regular (non-flagged) social media users as well as flagged, troll-like 
users. 
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5) Proportionately, Facebook has the greatest amount of junk news com- 
pared to other platforms, followed by Twitter. YouTube isa significant 
extreme news space generally, and a cultural commentary and debate 
space for “fake news' as issue. 

6) Dutch 4chan and Reddit circulate far more Dutch mainstream than 
junk news, with 4chan users likely commenting upon rather than 
taking over its narratives. 4chan is an incubator of far-right activ- 
ity in the Netherlands as seen through the types of YouTube videos 
referenced. 


Facebook: Fertile ground for junk news 


The method behind the research presented here derives from data journal- 
ism, particularly that of Buzzfeed News, and later the NRC Handelsblad, which 
both ascertained the most engaged-with stories on Facebook in the run-up 
to national elections. Whereas Buzzfeed News found that ‘fake news’ on 
Facebook outperformed mainstream news in the months preceding the US 
federal elections of 2016, leading in part to the overall ‘fake news crisis’, the 
NRC Handelsblad, deploying a far stricter definition, found scant presence 
of such material prior to the 2017 Dutch national elections. Our study found 
that Facebook is a fertile ground not for disinformation and fake news in 
the Netherlands but rather for junk news, a roomier definition than fake 
news as discussed above. Whilst it is not outperforming mainstream news, 
it is far from marginal, and in a few periods under study its presence in 
the top stories on Facebook (judged from engagement measures) equals or 
outperforms the mainstream. Engagement, measured by such interaction 
as shares and likes, requires further study, however, for it should not be 
equated in each case with support or agreement. 

While Facebook contains a significant stream of junk news, including 
hyperpartisan and conspiracy sources, foreign disinformation and fake 
news — such as organizations pretending to be news sources or advocacy 
groups — were found to be absent, at least for the most engaged-with stories 
related to the elections gathered through keyword queries of political lead- 
ers, parties and social issues (or in longer longitudinal studies of certain 
issue spaces on Twitter as well as in Dutch spaces on Reddit and 4chan). 
Disinformation and fake news may have not been encountered, but junk 
news is a factor in Dutch political spaces on social media and its impact 
should be critically studied. 
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Google Web Search: vernacular search queries result in junk news 


The Google Web Search study involved locating junk news within the top 
twenty results in Google.nl for queries of Dutch political parties and signifi- 
cant social issues prior to the Dutch provincial and European parliamentary 
elections of 2019. The queries were formulated by combining the names of 
the political parties with social issue keywords. The keywords derive both 
from the ‘official’ issue language collected from the party platforms as well 
as vernacular terms distilled from the comments on political party Facebook 
pages. Of the junk news found nearly all originate from hyperpartisan 
and tendentious sources rather than disinformation, conspiracy theory or 
clickbait. For particular groups of issue queries, up to 25% of the results were 
hyperpartisan. As on the other platforms under study, in the ‘top content’ 
no foreign disinformation, fake news organizations or fake advocacy groups 
were present during the election periods. 

The social issue keyword queries in combination with right-of-centre 
political parties resulted in junk news sources in greater quantities than 
that of left-of-centre party names. It was also found that the presence of junk 
news is not stable over time. Prior to the provincial elections the quantities 
rose, only to decline the day of the election and in its immediate aftermath, 
as witnessed by the issue of migration. The inverse was witnessed during 
the European parliamentary elections. When comparing the two types of 
search queries, the official and the vernacular, the latter results in a higher 
percentage of junk news in the results. 


Twitter: junk news and troll activity around polarizing subject 
matters 


The Twitter study examined the presence of junk news as well as troll-like 
activity during the campaign periods around the Dutch provincial and 
European parliamentary elections of 2019. There was troll-like activity 
encountered around the provincial elections around political terms such 
as the tag for the elections themselves (‘PS2019’), certain party leaders as 
well as potentially polarizing issues such as MH17, Zwarte Piet and the 
Utrecht tram shooting of March 2019. Troll-like activity refers to a series of 
behavioural indicators, including targeting politicians with unusually high 
bursts of tweets in short periods of time or through a set of accounts created 
at about the same time. The analysis found fourteen troll-like Twitter users 
were active around all issues studied and twenty-nine around most of them. 
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Four of these profiles remained active (or became active again) around the 
European parliamentary elections. They all spread mainly hyperpartisan 
and tendentious sources, followed in quantity by conspiracy websites. 

In answering the question of the extent to which junk news is present in 
the Dutch political Twittersphere, we ascertained the most shared sources, 
finding a steady resonance of junk news, paling in comparison however to 
mainstream sources. One finding of note is that during the Provincial elec- 
tions both Zwarte Piet and MH17 witnessed proportionately high amounts 
of activity, in spite of the fact that the Santa Klaus holiday (where Zwarte 
Piets make their appearance) does not take place until December and there 
was no particular MH17 news, for example concerning the investigations 
into the cause of the crash of the airliner. Troll-like users shared mostly 
hyperpartisan and tendentious sources, followed by conspiracy websites 
putting forward theories concerning MH17 and the Utrecht tram shooting. 
The pro-Russian site, novini.nl, which on a story level oscillates between 
hyperpartisan and conspiracy, also was circulated by troll-like users for 
all social issues under study, but only rarely in relation to political leaders. 
During the European Parliamentary elections junk news occasionally 
resonated more than mainstream news around such polarizing issues as 
Zwarte Piet and MH17. It outperformed mainstream news largely owing to 
the lack of news coverage of these issues during the periods under study, 
when junk news remains steady. 

Based on the findings, it appears that the Dutch Twittersphere does not 
have a disinformation problem; no professional or large-scale disinformation 
or fake advocacy campaigns were encountered. Troll-like users, whether 
in the form of bots, semi-automated users auto-retweeting and posting 
original content or unusually active users targeting politicians, do lend 
some symbolic power to divisive points of view around several social is- 
sues. Polarizing issue activity, fuelled by reference to hyperpartisan news, 
remains rather stable (albeit marginal) throughout both periods of study, 
suggesting that these issues do not resonate at expected times of the year 
only, but rather throughout. 


Instagram: a separate extreme media ecology and signs of 
artificial amplification 


The Instagram study inquired into the presence (and absence) of disinforma- 
tion and junk news in three ways: on a post-level, a source-level and that 
of followers. On a post-level, we examine the circulation of junk news in 
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political spaces, on a source-level we compare the audiences of junk news 
andpoliticalleaders and parties, and finally we study the follower bases of 
the political entities, searching for signs of inauthentic behaviour. In all we 
found a relatively healthy Dutch political arena on Instagram with only small 
amounts of junk news and fake followers. The vast majority of liked content 
in Dutch political Instagram, demarcated through political keyword queries, 
is not junk news, though around certain political leaders (at the far end of 
the political spectrum) and divisive issues small amounts of hyperpartisan 
news appear. Mainstream news was more prominent than junk news in the 
posts related to political parties and leaders in both periods under study. 
The most active users of the platform in the Dutch political Instagram arena 
are seemingly authentic with little sign artificial manipulation. 

Within this relatively healthy political space online the only suspicious 
activity encountered was on the far right of the political spectrum, where 
the circulation of junk news takes place and where those who follow the 
parties and leaders also follow the junk news sources. Here there are also 
indications of artificial manipulation. Certain party leaders (as well as 
the personal account of the Prime Minister) show signs of a significant 
fake follower base. The artificial activity found is in line with the 2015 fake 
follower incident when Twitter announced the deletion of fake followers 
that affected certain Dutch politicians (and celebrities) disproportionately. 


Reddit and 4chan: YouTube videos as news source contribute to 
polarization 


Despite their characterizations as alternative spaces on the web, the ‘Dutch’ 
Reddit and 4chan, following from our findings, do not appear to spread 
alternative news sources, but rather refer more often (even overwhelmingly) 
to mainstream news sources. Apart from witnessing examples of circulating 
the pro-Russian site, novini.nl, and the activity of one particularly suspect 
Reddit account, there does not appear to be any coordinated disinformation 
or fake news campaigning. There is certainly junk news to be found but 
compared to the overall spreading of sources the proportion is marginal. 
Particularly Reddit seems to be ‘resistant’ to disinformation. 

The research found the presence of junk news, especially of the hy- 
perpartisan variety, particularly in 4chan/pol/. These are largely links 
to tendentious and hyperpartisan sources such as The Post Online and 
De Dagelijkse Standaard, but also to the conspiracy site, NineForNews. A 
right-wing orientation was to be expected in gchan/pol/, given its reputation 


CONCLUSIONS 247 


as a hub of the extreme right, but it was perhaps less likely for Reddit, as 
the ‘Dutch’ Reddit that we took as a starting point contains a number of 
politically diverse subreddits. The articles that have gained the greatest 
salience concern migration and Islam rather than such geopolitical content 
as the Russian involvement in MH17. Whilst it may be problematic to label 
these sources as ‘fake’, they could be characterized as polarizing. 

Nevertheless, the vast majority of the links to news sources are directed 
at mainstream outlets such as NOS.nl, both on 4chan/pol as well as Reddit. 
These findings are counterintuitive in that the platforms are often described 
as alternative, as was said, and the anonymous and pseudonymous users 
point to marginal or alternative knowledge sources such as alternatives to 
Wikipedia. Our findings dispute such a characterization, for mainstream 
sources enjoy some authority on these platforms, but we did not as of yet 
research how they are discussed, e.g., as the starting point for a discussion 
or ridicule. On Reddit it could be that the mainstream NOS.nl is considered 
a reliable source and on 4chan/pol/ ‘fake news’. 

Finally, it is important not to regard mainstream and junk news as the 
only sources of news on the web. In both the Reddit and 4chan research 
but also in the YouTube study, we found that YouTube has emerged as a 
major news source. That can be said particularly for 4chan/pol/ but also for 
Reddit, where ‘News & Politics’ videos are a significant source. On 4chan/ 
pol/ they may be the most significant, quantitatively leaving other sources 
well behind. From a small explorative study of the YouTube channels posted 
on Reddit we found established sources referenced such as PowNed, Zondag 
met Lubach and NOS, while on 4chan/pol/ alternative, foreign and political 
(hyperpartisan) sources are pointed to, such as Mike Cernovich and Russia 
Today. Should such linking and engagement continue, such polarizing 
content could have a polarizing effect in the Dutch political space. 


From findings to implications: Mainstream under fire 


There is a small, but growing literature concerning how fake news could 
be considered a moral panic (Morozov, 2017; Hirst, 2017). The term refers to 
recurring episodes in history when ‘right-thinking people’ (defined seminally 
by Stanley Cohen as ‘editors, bishops and politicians’) spot a condition that is 
supposedly prompting a decline in societal standards and values (1972). When 
fake news is viewed through that lens, the concern is about how traditional 
journalism as a pillar or ‘4th estate’ of democracy is being hollowed out by 
social media and replaced by low-quality clickbait as well as openly ideological 
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commentary, both formatted in manners that drive their consumption 
not so unlike sugary junk food. The overall health of media as social fabric 
is said to be at stake, for citizens using social media as source for political 
information are disadvantaged in their capacity to form judgements about 
social issues and politics more broadly (Carlson, 2018). There is a second set of 
literature describing how the media coverage of so-called fake or junk news, 
and especially its relationship to the growth of an alternative, right-wing 
media ecology, gives it ‘oxygen’ (Phillips, 2018). More poignantly, it has been 
argued that journalistic coverage should turn its attention to the victims, 
rather than to the fascinating subcultural milieu online where the far right 
cultivates itself. There are also cases of politicians’ forwarding extremist and 
divisive content, which also gives it oxygen in the sense that it contributes 
to its spread and perhaps to its normalcy. Along all these lines, the recom- 
mendations concern identifying and acting upon threats to the mainstream, 
be they from social media platforms or from within the professions and 
practices of journalism, online content creation and political leadership. 
As we have found there are particular platforms and subject matters where 
the threats to the mainstream appear more acute. Whilst not a space where 
Dutch junk news sources are spread on a massive scale, the Dutch 4chan is 
an incubator of extremist sentiment, especially with respect to anti-Semitism 
and anti-immigration. Other platforms are problematic for different reasons. 
Dutch political spaces in Facebook and Twitter, demarcated through politi- 
cian, party and issue queries, have the largest quantities of junk news that is 
engaged with, though they are still smaller than mainstream news consump- 
tion overall in those same spaces. Among the junk news, hyperpartisan 
sources (rather than disinformation or conspiracy) are amongst the more 
popular, and for divisive subject matters such as climate change, MH17 and 
Zwarte Piet their stories occasionally outperform those in the mainstream 
press. On Twitter during the European parliamentary election campaign 
period, for example, a pro-Nexit story in the hyperpartisan newspaper, De 
Dagelijkse Standaard, about the Netherlands leaving the EU outperformed 
a counterpart article in the mainstream NRC Handelsblad. A more general 
polarized media ecology is also in evidence. On YouTube an alternative media 
sphere has formed, where extreme YouTubers, or micro-celebrities, hold 
sway. Instagram also has a new-right, alternative media space, analytically 
detected through shared followers of politicians at the far end of the political 
spectrum and hyperpartisan media organizations. These are largely ‘alt lite’, 
meaning anti-establishment and anti-political correctness, with content 
that also could be considered anti-Islam. There are no discernible left-wing 
equivalents. Rather, these spaces compete with more mainstream ones. 
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In contrast to the situation in other countries during the European Par- 
liamentary elections, in our study we did not find foreign disinformation 
but rather so-called junk news, especially around particular issues, such 
as Zwarte Piet, MH17, climate and the European Union (Peel, 2019). We 
also found it around the topic of ‘fake news’, studied in this instance as a 
social issue. Although decent quantities of junk news were in evidence, 
mainstream news largely outperformed it. The largest quantities of junk 
news circulated not so much around political parties and leaders (with 
some exceptions), but around specific polarizing issues. Junk news activity 
around these issues sometimes appeared during the election periods, but 
for other issues there was year-round activity, even for such seasonal issues 
as Zwarte Piet. Thus, the question is not only whether there is junk news 
around election time, but also more generally when it manifests itself, and 
with which intensity and duration. 

The following policy implications of our work are directed specifically 
at the phenomenon of junk news, rather than at foreign disinformation 
and fake news from organizations feigning to be news organizations or 
fake advocacy groups, of which we found none, at least in the top or most 
engaged-with content related to Dutch politics across the web and social 
media platforms. Our recommendations concern the recognition and 
monitoring of the polarization of the media landscape, the devitalization 
or disengagement with extreme content, a national conversation about issues 
that appear frequently in junk news (such as Zwarte Piet) rather than one 
about disinformation or fake news generally, training for professionals that 
produce online content, and enabling access to the (increasingly inaccessible) 
data on social media platforms for research and media monitoring. 


Policy themes in brief 


1) The monitoring of the polarization of the media landscape, and the 
mainstreaming of polarizing media with extreme content on social 
media platforms. 


Social media platforms rely on software, their users as well as content 
reviewers to detect extreme content. More and more of it is subsequently 
removed. But historically the attention paid by social media companies 
to extreme content has been uneven, and definitions unstable. It thereby 
remains desirable to institute independent monitoring. Such work could 
be taken up by academic researchers, non-governmental organizations, 
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governmental agencies specialized in extremism and polarization as well 
as media watchdogs. 


2) Media training for professional content makers — from journalists to 
digital media producers — concerning online source criticism as well 
as amplification or ‘oxygen-giving’ of extreme speech actors in society. 


The Netherlands has existing media literacy training programs, designed for 
example for senior citizens as well as primary and secondary school students. 
This recommendation is made specifically for professional content-makers 
such as journalists and editors. It could be made a part of existing or new 
media literacy programs dedicated to online source criticism and dealing 
with polarizing content (see also point three below). Such a training program 
is also of use to lecturers in higher education, policymakers and civil society. 


3) No oxygen-giving to extreme actors and their (online) content by media 
organizations. 


In our study we found that tendentious news stories circulate well during 
the election campaign periods and beyond. The articles are shared and liked 
by troll-like users but also by regular news consumers, which we found for 
example on Twitter during the Provincial elections. In the same spaces we 
also found users sharing and liking discriminatory, anti-Semitic, misogynist 
and xenophobic content, albeit it to a lesser degree. 

It is important not to equate tendentious and extreme media, even when 
they appear to share standpoints without using the same words. Similarly, 
that tendentious media is on the rise and mainstreaming does not mean 
that similar weight should be given to extreme media. The recommenda- 
tion is that no oxygen should be given to extreme media sources and their 
content, meaning no sharing, liking, reacting, commenting, retweeting or 
YouTube-debating. Any form of engagement with such content increases 
the attention and the metrics and contributes to its spread, ranking and 
normalization. Such a recommendation goes for public broadcasting and 
commercial media organizations, but also for the tendentious media. Instead 
of journalists’ writing about far-right subcultures, attention could be spent 
on their victims (Philips, 2018). 


4) Recognition of polarizing issues such as Zwarte Piet and the facilitation 
of national and regional conversations. 
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The research found that attention to polarizing issues such as Zwarte Piet is 
year-round rather than seasonal. Such recognition of increasing polarization 
in society should lead to discussions about how common ground may be 
found. The Netherlands has a tradition of collective discussion concerning 
major societal issues through such mechanisms as the Brede Maatschap- 
pelijke Debat (society-wide debate) and interactive policy making. There are 
other contemporary forms of citizen participation and discussion that could 
be instrumental in dealing with polarizing issues and cultural contestation. 
Institutions experienced in organizing societal discussion and debate should 
be called upon and supported to do so, and bottom-up initiatives should 
be facilitated. 


5) Advocacy for social media data access for researchers, journalists and 
watchdogs, and creation of research archives of deleted content. 


The current issue of ‘locked platforms’ concerns the extent to which social 
media companies are making their data inaccessible to researchers, journal- 
ists and non-governmental organizations. As an answer to governmental 
concern about ‘dark political posts’ (political ads directed only at a segment 
of users in their newsfeeds) and other political ads without clear provenance, 
Facebook has launched a political ad archive tool and API. But at the same 
time Facebook has removed in part or in whole access to services such as 
the Pages API and Graph Search, which had been in widespread use by 
researchers. Social media companies should take up the task of making 
available the data that researchers, journalists and non-governmental 
organizations would like to use for the purposes of research, monitoring 
and archiving. Governmental agencies, in consultation with the users and 
use types mentioned above, have a facilitative as well as a regulatory role 
to play here. 
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9 Epilogue 
After the tweet storm 


Richard Rogers and Sal Hagen 


Abstract 

The publication of the study elicited reactions, especially on Twitter, where 
questions arose about the use of the notion of junk news, rather than ‘pulp 
news’, among other points. The analogy to junk food is emphasised. There 
was also the question of symmetry, and the treatment of both ends of the 
political spectrum. Why is the new populist right identified as the pur- 
veyors of extreme content? We found a polarised Dutch media landscape 
where hyperpartisan (and to a lesser extent conspiracy) content from new 
populist right (rather than the left or other orientations) circulates well 
on social media. Unlike in the US during the initial Trump insurgency, 
mainstream news in the Netherlands still outperforms what was hitherto 
known as ‘fake news’, across all platforms. 


Keywords: Twitter, new populist right, junk news, mainstream news, 
Dutch media landscape 


In line with the theme of our study, ‘The Politics of Social Media Manipula- 
tion’, commissioned by the Ministry of Internal Affairs and published in 
October 2019, most of the attention it has garnered since has been on social 
media rather than in the press. One reason is our report into so-called fake 
news and disinformation, was published on a Friday afternoon prior to the 
October holiday break. It also contains no scintillating findings concern- 
ing (Russian or foreign) disinformation campaigning in the Netherlands; 
during the run up to the two elections in 2019 no disinformation, front 
groups or fake news sources were found whose stories were circulating on 
Facebook, Instagram, Twitter, Google Web Search, YouTube, Reddit or 4chan, 
the multiple platforms under study. Since there was no disinformation in 
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circulation, at least not any that Facebook and other social media users 
engaged with in any significant degree, could the ‘Echt?!’ fake news public 
awareness campaign the Ministry ran be considered tilting at windmills? Is 
the study and monitoring of fake news and disinformation in the Netherlands 
a worthwhile endeavour, given that to date hardly any has been found? 

These are some of the questions that have arisen in the one space where 
there has been considerable commentary on the study, Twitter. The reason 
is that the report found the rise of a growing alternative, new right media 
sphere in the Netherlands, without an equivalent on the left, which also 
manifests itself on Twitter. This media sphere contains stories (and sources) 
that are hyperpartisan, conspiracy-related and/or clickbait, which the 
study collectively defines as ‘junk news’, employing a term used by Oxford 
University researchers characterizing similar sources elsewhere. Junk 
news shares commonalities with junk food, in that it contains attractively 
packaged stories, of a low journalistic standard, that we know we probably 
should not consume, such as sensationalist clickbait, headlines and teaser 
texts designed to be clicked rather than actually read. 

Especially on Facebook a great deal of such material circulated prior to the 
elections, and was liked or shared, tallying high engagement scores. Through 
the circulation of their stories on Facebook, particularly hyperpartisan 
sites, defined as ‘openly ideological web operations’, are on the rise in the 
Netherlands, certainly compared to the results of a smaller study published 
in the NRC Handelsblad on political stories that circulated on Facebook 
prior to the 2017 national elections, when there were relatively few. For 
example, the hyperpartisan site, De Dagelijkse Standaard, received more 
attention during both 2019 election periods than the mainstream media 
sites, RTL Nieuws and NU.nl. Openly extreme sites such as Fenixx nearly had 
the same engagement on Facebook for political news stories as the quality 
newspaper, the NRC Handelsblad. Hyperpartisan, left-wing sites with similar 
engagement scores were hardly in evidence. It should be pointed out that 
mainstream news, in total, still outperforms alternative, new-right (or, using 
the terminology of political scientist, Cas Mudde, ‘populist radical right’) 
sources for election-related subjects (Bahara, 2019), but not for every issue 
and not for every election period under study. For example, we found that 
whilst political issues, parties and leaders may wax and wane, there are 
particular issues in this alternative media sphere that are hot year-round, 
such as Zwarte Piet, climate change and the European Union itself. Both the 
high engagement scores for hyperpartisan content as well as the unceasing 
attention to divisive issues by the alternative media sphere could be viewed 
as indications of societal unrest. 
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Social media such as Twitter and the online comment space more broadly 
defined are sometimes described as increasingly toxic, and commentators as 
more and more uninhibited; indeed, hundreds of the online reactions to our 
study could be characterized as such. We also found, in the reactions, that 
there are (albeit very few) users who behave troll-like, active across divisive 
issues and ever-targeting politicians, and are actually battling misinformation. 

In the recommendations of the report, which we entitled ‘the mainstream 
under fire’, we called for the (academic, non-governmental) monitoring of the 
growing polarization in the country that we identified in the rise of new-right 
alternative media. We also believe that continued study of the mainstreaming 
of hyperpartisan and extreme content remains important here, even if it 
is not the product of organized influence campaigning by foreign actors. 
For example, certain Dutch hyperpartisan sources are actively seeking to 
mainstream by adding newswire stories to supplement the pages of their 
otherwise fervent commentary. Through the circulation on Facebook and 
other social media as well as its large-scale engagement through likes and 
shares, certain extreme content also is gaining more exposure. We recommend 
that these stories, whether brought together with actual news or otherwise 
shared even if in gest, should not be given so much oxygen. Right-of-centre, 
self-described tendentious media sources but also other mainstream and 
transgressive media should consider refraining from rebroadcasting extreme 
content. Finally, we note that Facebook (and Instagram) have ceased allowing 
researcher access to its (API page) data, making such studies as ours increas- 
ingly arduous to undertake. Considering the important cultural and societal 
stakes, we need to be able to study Facebook and other social media platform 
data to understand the nature and scale of the problem now and in future. 
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